mysql 查询卡在状态 "sending data"
mysql query stuck with state "sending data"
在几乎需要 7 的查询下面，我对它在 mysqlserver（ndb 存储引擎）中的正常行为感到困惑，在解释中它还显示它正在使用 table 索引
SELECT radgroupreply.groupname,
count(distinct(radusergroup.username)) AS users
FROM radgroupreply
JOIN radusergroup ON radgroupreply.groupname=radusergroup.groupname
WHERE
(radgroupreply.groupname NOT LIKE 'FB-%'
AND radgroupreply.groupname <> 'Dropped Corporate Users'
AND radgroupreply.groupname <> 'Dropped Broadband Users')
GROUP BY radgroupreply.groupname
UNION
SELECT distinct(radgroupcheck.groupname),
count(distinct(radusergroup.username))
FROM radgroupcheck
JOIN radusergroup ON radgroupcheck.groupname=radusergroup.groupname
WHERE
(radgroupcheck.groupname NOT LIKE 'FB-%'
AND radgroupcheck.groupname <> 'Dropped Corporate Users'
)
GROUP BY radgroupcheck.groupname ORDER BY groupname asc;
查询的 explain 输出是：-
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+---------------------------------+
| 1 | PRIMARY | radgroupreply | range | groupname | groupname | 66 | NULL | 47 | Using where; Using MRR |
| 1 | PRIMARY | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupreply.groupname | 64 | NULL |
| 2 | UNION | radgroupcheck | range | groupname | groupname | 66 | NULL | 20 | Using where; Using MRR |
| 2 | UNION | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupcheck.groupname | 120 | NULL |
|NULL| UNION RESULT | <union1,2> | ALL | NULL | NULL | NULL | NULL | NULL | Using temporary; Using filesort |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+---------------------------------+
下面是 table 结构和索引列信息 table 参与 join
Table: radgroupreply; # 总共 192 行
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| groupname | varchar(64) | NO | MUL | | |
| attribute | varchar(32) | NO | | | |
| op | char(2) | NO | | = | |
| value | varchar(253) | NO | | | |
+-----------+------------------+------+-----+---------+----------------+
Table：radusergroup #total：~13000 行
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| username | varchar(64) | NO | MUL | | |
| groupname | varchar(64) | NO | MUL | | |
| priority | int(11) | NO | | 1 | |
+-----------+-------------+------+-----+---------+----------------+
Table: radgroupcheck #totalrows: ~ 100
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| groupname | varchar(64) | NO | MUL | | |
| attribute | varchar(32) | NO | | | |
| op | char(2) | NO | | == | |
| value | varchar(253) | NO | | | |
+-----------+------------------+------+-----+---------+----------------+
#radusergroup# CREATE TABLE `radusergroup` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(64) NOT NULL DEFAULT '',
`groupname` varchar(64) NOT NULL DEFAULT '',
`priority` int(11) NOT NULL DEFAULT '1',
PRIMARY KEY (`id`),
KEY `groupname` (`groupname`),
KEY `username` (`username`)
) ENGINE=ndbcluster AUTO_INCREMENT=12380 DEFAULT CHARSET=latin1
#show index from radusergroup
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radgroupreply | 0 | PRIMARY | 1 | id | A | 192 | NULL | NULL | | BTREE | | |
| radgroupreply | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
#radgroupreply# CREATE TABLE `radgroupreply` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`groupname` varchar(64) NOT NULL DEFAULT '',
`attribute` varchar(32) NOT NULL DEFAULT '',
`op` char(2) NOT NULL DEFAULT '=',
`value` varchar(253) NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `groupname` (`groupname`)
) ENGINE=ndbcluster AUTO_INCREMENT=2410 DEFAULT CHARSET=latin1
mysql> show index from radgroupreply;
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radgroupreply | 0 | PRIMARY | 1 | id | A | 192 | NULL | NULL | | BTREE | | |
| radgroupreply | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
#radgroupcheck# CREATE TABLE `radgroupcheck` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`groupname` varchar(64) NOT NULL DEFAULT '',
`attribute` varchar(32) NOT NULL DEFAULT '',
`op` char(2) NOT NULL DEFAULT '==',
`value` varchar(253) NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `groupname` (`groupname`)
) ENGINE=ndbcluster AUTO_INCREMENT=588 DEFAULT CHARSET=latin1
mysql> show index from radgroupcheck;
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radgroupcheck | 0 | PRIMARY | 1 | id | A | 103 | NULL | NULL | | BTREE | | |
| radgroupcheck | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> SELECT @@GLOBAL.tx_isolation, @@tx_isolation, @@session.tx_isolation;
+-----------------------+----------------+------------------------+
| @@GLOBAL.tx_isolation | @@tx_isolation | @@session.tx_isolation |
+-----------------------+----------------+------------------------+
| READ-COMMITTED | READ-COMMITTED | READ-COMMITTED |
+-----------------------+----------------+------------------------+
更新：查询
(SELECT radgroupreply.groupname,
count(distinct(radusergroup.username)) AS users
FROM radgroupreply
JOIN radusergroup ON radgroupreply.groupname=radusergroup.groupname
WHERE
(radgroupreply.groupname NOT LIKE 'FB-%' AND radgroupreply.groupname <> 'Dropped Corporate Users' AND radgroupreply.groupname <> 'Dropped Broadband Users')
GROUP BY radgroupreply.groupname )
UNION
(SELECT radgroupcheck.groupname,
count(distinct(radusergroup.username))
FROM radgroupcheck
JOIN radusergroup ON radgroupcheck.groupname=radusergroup.groupname
WHERE
(radgroupcheck.groupname NOT LIKE 'FB-%' AND radgroupcheck.groupname <> 'Dropped Corporate Users')
GROUP BY radgroupcheck.groupname ORDER BY groupname asc);
解释：-
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+------------------------+
| 1 | PRIMARY | radgroupreply | range | groupname | groupname | 66 | NULL | 47 | Using where; Using MRR |
| 1 | PRIMARY | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupreply.groupname | 64 | NULL |
| 2 | UNION | radgroupcheck | range | groupname | groupname | 66 | NULL | 20 | Using where; Using MRR |
| 2 | UNION | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupcheck.groupname | 121 | NULL |
|NULL| UNION RESULT | <union1,2> | ALL | NULL | NULL | NULL | NULL | NULL | Using temporary |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+------------------------+
因为您已经是 GROUPing BY，所以该列将 SELECT distinct(radgroupcheck.groupname) 更改为 SELECT radgroupcheck.groupname
添加括号以说明 UNION 正在排序，而不仅仅是最后一个 SELECT:
( SELECT ... ) UNION ( SELECT ... ) ORDER BY ...
根据提示here修改many:manytable（引擎除外）。
group_reply 和 group_check 似乎具有相同的模式；他们分开的原因是什么？
group_reply 和 group_check 中是否有可用于 PRIMARY KEY 的唯一列（或一对列）？
改造
SELECT groupname,
( SELECT count(distinct username)
FROM radusergroup WHERE groupname = u.groupname
) AS users
FROM (
(
SELECT r.groupname
FROM radgroupreply AS r
WHERE r.groupname NOT LIKE 'FB-%'
AND r.groupname <> 'Dropped Corporate Users'
AND r.groupname <> 'Dropped Broadband Users'
)
UNION DISTINCT -- or UNION ALL ?
(
SELECT c.groupname
FROM radgroupcheck AS c
WHERE c.groupname NOT LIKE 'FB-%'
AND c.groupname <> 'Dropped Corporate Users'
)
) AS u
ORDER BY groupname asc
我会将该查询重写为这个查询：
SELECT radgroupreply.groupname,
count(distinct radusergroup.username ) AS users,
count(distinct CASE
WHEN radgroupreply.groupname <> 'Dropped Broadband Users'
THEN radusergroup.username
END ) As users_without_dropped_broadband
FROM radgroupreply
JOIN radusergroup ON radgroupreply.groupname=radusergroup.groupname
WHERE
(radgroupreply.groupname NOT LIKE 'FB-%'
AND radgroupreply.groupname <> 'Dropped Corporate Users' )
GROUP BY radgroupreply.groupname
ORDER BY groupname asc
;
以上查询以略有不同的形式显示结果：
| gropname | users | users_without_dropped_broadband |
| A | 20 | 13 |
| B | 33 | 11 |
比原始查询的布局：
| gropname | users |
| A | 20 |
| A | 13 |
| B | 33 |
| B | 11 |
但是这个新查询执行所有操作（读取表、连接、计数等）只执行一次，
不是原始查询的两次，应该是至少快 50%（<3.5 秒）。
在几乎需要 7 的查询下面，我对它在 mysqlserver（ndb 存储引擎）中的正常行为感到困惑，在解释中它还显示它正在使用 table 索引
SELECT radgroupreply.groupname,
count(distinct(radusergroup.username)) AS users
FROM radgroupreply
JOIN radusergroup ON radgroupreply.groupname=radusergroup.groupname
WHERE
(radgroupreply.groupname NOT LIKE 'FB-%'
AND radgroupreply.groupname <> 'Dropped Corporate Users'
AND radgroupreply.groupname <> 'Dropped Broadband Users')
GROUP BY radgroupreply.groupname
UNION
SELECT distinct(radgroupcheck.groupname),
count(distinct(radusergroup.username))
FROM radgroupcheck
JOIN radusergroup ON radgroupcheck.groupname=radusergroup.groupname
WHERE
(radgroupcheck.groupname NOT LIKE 'FB-%'
AND radgroupcheck.groupname <> 'Dropped Corporate Users'
)
GROUP BY radgroupcheck.groupname ORDER BY groupname asc;
查询的 explain 输出是：-
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+---------------------------------+
| 1 | PRIMARY | radgroupreply | range | groupname | groupname | 66 | NULL | 47 | Using where; Using MRR |
| 1 | PRIMARY | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupreply.groupname | 64 | NULL |
| 2 | UNION | radgroupcheck | range | groupname | groupname | 66 | NULL | 20 | Using where; Using MRR |
| 2 | UNION | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupcheck.groupname | 120 | NULL |
|NULL| UNION RESULT | <union1,2> | ALL | NULL | NULL | NULL | NULL | NULL | Using temporary; Using filesort |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+---------------------------------+
下面是 table 结构和索引列信息 table 参与 join
Table: radgroupreply; # 总共 192 行
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| groupname | varchar(64) | NO | MUL | | |
| attribute | varchar(32) | NO | | | |
| op | char(2) | NO | | = | |
| value | varchar(253) | NO | | | |
+-----------+------------------+------+-----+---------+----------------+
Table：radusergroup #total：~13000 行
+-----------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+-------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| username | varchar(64) | NO | MUL | | |
| groupname | varchar(64) | NO | MUL | | |
| priority | int(11) | NO | | 1 | |
+-----------+-------------+------+-----+---------+----------------+
Table: radgroupcheck #totalrows: ~ 100
+-----------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------+------+-----+---------+----------------+
| id | int(11) unsigned | NO | PRI | NULL | auto_increment |
| groupname | varchar(64) | NO | MUL | | |
| attribute | varchar(32) | NO | | | |
| op | char(2) | NO | | == | |
| value | varchar(253) | NO | | | |
+-----------+------------------+------+-----+---------+----------------+
#radusergroup# CREATE TABLE `radusergroup` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`username` varchar(64) NOT NULL DEFAULT '',
`groupname` varchar(64) NOT NULL DEFAULT '',
`priority` int(11) NOT NULL DEFAULT '1',
PRIMARY KEY (`id`),
KEY `groupname` (`groupname`),
KEY `username` (`username`)
) ENGINE=ndbcluster AUTO_INCREMENT=12380 DEFAULT CHARSET=latin1
#show index from radusergroup
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radgroupreply | 0 | PRIMARY | 1 | id | A | 192 | NULL | NULL | | BTREE | | |
| radgroupreply | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
#radgroupreply# CREATE TABLE `radgroupreply` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`groupname` varchar(64) NOT NULL DEFAULT '',
`attribute` varchar(32) NOT NULL DEFAULT '',
`op` char(2) NOT NULL DEFAULT '=',
`value` varchar(253) NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `groupname` (`groupname`)
) ENGINE=ndbcluster AUTO_INCREMENT=2410 DEFAULT CHARSET=latin1
mysql> show index from radgroupreply;
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radgroupreply | 0 | PRIMARY | 1 | id | A | 192 | NULL | NULL | | BTREE | | |
| radgroupreply | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
#radgroupcheck# CREATE TABLE `radgroupcheck` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`groupname` varchar(64) NOT NULL DEFAULT '',
`attribute` varchar(32) NOT NULL DEFAULT '',
`op` char(2) NOT NULL DEFAULT '==',
`value` varchar(253) NOT NULL DEFAULT '',
PRIMARY KEY (`id`),
KEY `groupname` (`groupname`)
) ENGINE=ndbcluster AUTO_INCREMENT=588 DEFAULT CHARSET=latin1
mysql> show index from radgroupcheck;
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| radgroupcheck | 0 | PRIMARY | 1 | id | A | 103 | NULL | NULL | | BTREE | | |
| radgroupcheck | 1 | groupname | 1 | groupname | A | NULL | NULL | NULL | | BTREE | | |
+---------------+------------+-----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
mysql> SELECT @@GLOBAL.tx_isolation, @@tx_isolation, @@session.tx_isolation;
+-----------------------+----------------+------------------------+
| @@GLOBAL.tx_isolation | @@tx_isolation | @@session.tx_isolation |
+-----------------------+----------------+------------------------+
| READ-COMMITTED | READ-COMMITTED | READ-COMMITTED |
+-----------------------+----------------+------------------------+
更新：查询
(SELECT radgroupreply.groupname,
count(distinct(radusergroup.username)) AS users
FROM radgroupreply
JOIN radusergroup ON radgroupreply.groupname=radusergroup.groupname
WHERE
(radgroupreply.groupname NOT LIKE 'FB-%' AND radgroupreply.groupname <> 'Dropped Corporate Users' AND radgroupreply.groupname <> 'Dropped Broadband Users')
GROUP BY radgroupreply.groupname )
UNION
(SELECT radgroupcheck.groupname,
count(distinct(radusergroup.username))
FROM radgroupcheck
JOIN radusergroup ON radgroupcheck.groupname=radusergroup.groupname
WHERE
(radgroupcheck.groupname NOT LIKE 'FB-%' AND radgroupcheck.groupname <> 'Dropped Corporate Users')
GROUP BY radgroupcheck.groupname ORDER BY groupname asc);
解释：-
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+------------------------+
| 1 | PRIMARY | radgroupreply | range | groupname | groupname | 66 | NULL | 47 | Using where; Using MRR |
| 1 | PRIMARY | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupreply.groupname | 64 | NULL |
| 2 | UNION | radgroupcheck | range | groupname | groupname | 66 | NULL | 20 | Using where; Using MRR |
| 2 | UNION | radusergroup | ref | groupname | groupname | 66 | ctradius.radgroupcheck.groupname | 121 | NULL |
|NULL| UNION RESULT | <union1,2> | ALL | NULL | NULL | NULL | NULL | NULL | Using temporary |
+----+--------------+---------------+-------+---------------+-----------+---------+----------------------------------+------+------------------------+
因为您已经是 GROUPing BY，所以该列将 SELECT distinct(radgroupcheck.groupname) 更改为 SELECT radgroupcheck.groupname
添加括号以说明 UNION 正在排序，而不仅仅是最后一个 SELECT:
( SELECT ... ) UNION ( SELECT ... ) ORDER BY ...
根据提示here修改many:manytable（引擎除外）。
group_reply 和 group_check 似乎具有相同的模式；他们分开的原因是什么？
group_reply 和 group_check 中是否有可用于 PRIMARY KEY 的唯一列（或一对列）？
改造
SELECT groupname,
( SELECT count(distinct username)
FROM radusergroup WHERE groupname = u.groupname
) AS users
FROM (
(
SELECT r.groupname
FROM radgroupreply AS r
WHERE r.groupname NOT LIKE 'FB-%'
AND r.groupname <> 'Dropped Corporate Users'
AND r.groupname <> 'Dropped Broadband Users'
)
UNION DISTINCT -- or UNION ALL ?
(
SELECT c.groupname
FROM radgroupcheck AS c
WHERE c.groupname NOT LIKE 'FB-%'
AND c.groupname <> 'Dropped Corporate Users'
)
) AS u
ORDER BY groupname asc
我会将该查询重写为这个查询：
SELECT radgroupreply.groupname,
count(distinct radusergroup.username ) AS users,
count(distinct CASE
WHEN radgroupreply.groupname <> 'Dropped Broadband Users'
THEN radusergroup.username
END ) As users_without_dropped_broadband
FROM radgroupreply
JOIN radusergroup ON radgroupreply.groupname=radusergroup.groupname
WHERE
(radgroupreply.groupname NOT LIKE 'FB-%'
AND radgroupreply.groupname <> 'Dropped Corporate Users' )
GROUP BY radgroupreply.groupname
ORDER BY groupname asc
;
以上查询以略有不同的形式显示结果：
| gropname | users | users_without_dropped_broadband |
| A | 20 | 13 |
| B | 33 | 11 |
比原始查询的布局：
| gropname | users |
| A | 20 |
| A | 13 |
| B | 33 |
| B | 11 |
但是这个新查询执行所有操作（读取表、连接、计数等）只执行一次，
不是原始查询的两次，应该是至少快 50%（<3.5 秒）。