mySQL >> 使用 CONCAT、GROUP_CONCAT 和多个 JOINS 的查询非常慢（63 条记录需要 100 秒）
mySQL >> Very Slow Query with CONCAT, GROUP_CONCAT and multiple JOINS (100 sec for 63 records)
我正在使用以下查询：
SELECT *,
GROUP_CONCAT(distinct t_name order by t_id separator ', ') 'Topics',
GROUP_CONCAT(distinct ch_title order by ch_id separator ', ') 'Chapters',
GROUP_CONCAT(distinct cs_code order by cs_id separator ', ') 'ContentStandards',
GROUP_CONCAT(distinct c_name order by c_id separator ', ') 'Categories',
GROUP_CONCAT(distinct s_name order by s_id separator ', ') 'SchoolSubjects'
FROM (
SELECT r.res_id, r.res_status, r.res_type, r.res_category, r.res_title, r.res_filename, r.res_source, r.res_gradelevel_from, r.res_gradelevel_to, r.res_tags, r.res_description, r.res_onserver, r.res_favoriteaccts, r.res_CreatedOn, c_id, s_id, t_id, ch_id, cs_id,
CONCAT("<a class=""ReportLink"" href=""page.asp?category=",c_name,""" title=""",c_name,""">",c_name,"</a>") as c_name,
CONCAT("<a class=""ReportLink"" href=""page.asp?subject=",s_name,""" title=""",s_name,""">",s_name,"</a>") as s_name,
CONCAT("<a class=""ReportLink"" href=""page.asp?topic=",t_name,""" title=""",t_name,""">",t_name,"</a>") as t_name,
CONCAT("<a class=""ReportLink"" href=""page.asp?cs=",cs_code,""" title=""",cs_code,""">",cs_code,"</a>") as cs_code,
CONCAT("<a class=""ReportLink"" href=""page.asp?chapter=",ch_title,""" title=""",ch_title,""">",ch_code,": ",ch_title,"</a>") as ch_title
FROM hre_resources r
LEFT JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
LEFT JOIN topics t on t.t_id = ttr.rt_topicid
LEFT JOIN topic_to_cs tcs on tcs.cst_topicid = t.t_id
LEFT JOIN categories c ON c_id = r.res_category
LEFT JOIN topic_to_framework ttc on ttc.ft_topicid = t.t_id
LEFT JOIN ca_hss_frameworkchapters ch ON ch.ch_id = ttc.ft_frameworkid
LEFT JOIN ca_hss_contentstandards cs ON cs.cs_id = tcs.cst_contentstandardid
LEFT JOIN schoolsubjects_to_resource str ON str.sr_resid = r.res_id
LEFT JOIN schoolsubjects s ON s.s_id = str.sr_sid) links
WHERE RES_Status = 'Active'
GROUP BY RES_ID
ORDER BY RES_Title ASC, RES_Source DESC
此查询 returns 来自我们 mySQL 数据库的 64 条格式完美的记录，但执行此操作需要 100 多秒。
查询分析器将主要违规者显示为：
+-------------------------------+-----------+
| Sending data | 34.530026 |
+-------------------------------+-----------+
| Creating sort index | 24.205878 |
| Storing result in query cache | 56.739204 |
+-------------------------------+-----------+
解释结果如下：
+----+-------------+------------+--------+---------------+---------------+---------+------------------------------------+------+---------------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+---------------+---------------+---------+------------------------------------+------+---------------------------------------------------------------------+
| 1 | PRIMARY | <derived2> | ref | <auto_key0> | <auto_key0> | 77 | const | 10 | Using index condition; Using where; Using temporary; Using filesort |
| 2 | DERIVED | r | ALL | NULL | NULL | NULL | NULL | 64 | NULL |
| 2 | DERIVED | ttr | ref | RT_ResourceID | RT_ResourceID | 4 | uhri_hre.r.RES_ID | 3 | Using where |
| 2 | DERIVED | t | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.ttr.RT_TopicID | 1 | Using where |
| 2 | DERIVED | tcs | ref | CST_TopicID | CST_TopicID | 4 | uhri_hre.t.T_ID | 9 | Using where |
| 2 | DERIVED | c | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.r.RES_Category | 1 | Using where |
| 2 | DERIVED | ttc | ref | FT_TopicID | FT_TopicID | 4 | uhri_hre.t.T_ID | 21 | Using where |
| 2 | DERIVED | ch | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.ttc.FT_FrameworkID | 1 | Using where |
| 2 | DERIVED | cs | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.tcs.CST_ContentStandardID | 1 | Using where |
| 2 | DERIVED | str | ref | SR_RESID | SR_RESID | 4 | uhri_hre.r.RES_ID | 1 | Using where |
| 2 | DERIVED | s | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.str.SR_SID | 1 | Using where |
+----+-------------+------------+--------+---------------+---------------+---------+------------------------------------+------+---------------------------------------------------------------------+
这些表具有以下索引。未列出的表在 ID 字段（主键）上有索引：
HRE_Resources:
+--------------+-------+--------+--------+--------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+--------------+-------+--------+--------+--------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | RES_ID | 64 | A | | |
| RES_Type | BTREE | No | No | RES_Type | 4 | A | | |
| RES_Category | BTREE | No | No | RES_Category | 10 | A | | |
| RES_Status | BTREE | No | No | RES_Status | 4 | A | | |
| RES_Language | BTREE | No | No | RES_Language | 8 | A | | |
| RES_Source | BTREE | No | No | RES_Source | 21 | A | | |
+--------------+-------+--------+--------+--------------+-------------+-----------+------+---------+
Topics:
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | T_ID | 39 | A | | |
| T_Name | BTREE | No | No | T_Name | 39 | A | | |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
Topic_to_CS:
+-----------------------+-------+--------+--------+-----------------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+-----------------------+-------+--------+--------+-----------------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | CST_ID | 685 | A | | |
| CST_ContentStandardID | BTREE | No | No | CST_ContentStandardID | 228 | A | | |
| CST_TopicID | BTREE | No | No | CST_TopicID | 76 | A | | |
+-----------------------+-------+--------+--------+-----------------------+-------------+-----------+------+---------+
Topic_to_Framework:
+----------------+-------+--------+--------+----------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+----------------+-------+--------+--------+----------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | FT_ID | 471 | A | | |
| FT_TopicID | BTREE | No | No | FT_TopicID | 22 | A | | |
| FT_FrameworkID | BTREE | No | No | FT_FrameworkID | 235 | A | | |
+----------------+-------+--------+--------+----------------+-------------+-----------+------+---------+
Topic_to_Resource:
+---------------+-------+--------+--------+---------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+---------------+-------+--------+--------+---------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | RT_ID | 387 | A | | |
| RT_ResourceID | BTREE | No | No | RT_ResourceID | 129 | A | | |
| RT_TopicID | BTREE | No | No | RT_TopicID | 77 | A | | |
+---------------+-------+--------+--------+---------------+-------------+-----------+------+---------+
Schoolsubject_to_Resource:
+----------+-------+--------+--------+----------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+----------+-------+--------+--------+----------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | SR_ID | 228 | A | | |
| SR_SID | BTREE | No | No | SR_SID | 25 | A | | |
| SR_RESID | BTREE | No | No | SR_RESID | 228 | A | | |
+----------+-------+--------+--------+----------+-------------+-----------+------+---------+
Categories:
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | C_ID | 9 | A | | |
| C_Name | BTREE | No | No | C_Name | 9 | A | | |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
其中一些索引是使用其他查询中使用的更复杂的 WHERE 创建的。
出于其他目的，我对数据库进行了 运行 复杂查询，这些查询与上述查询无关，而且它的执行名义上是正常的。因此，我认为这不是服务器资源问题。
任何人都可以在上面的查询中看到任何可能导致响应如此缓慢的内容吗？
非常感谢！
PS。根据以下请求从服务器添加一些信息...
+------------------------------+---------+
| Variable_name | Value |
+------------------------------+---------+
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 1048576 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
+------------------------------+---------+
+-------------------------+--------+
| Variable_name | Value |
+-------------------------+--------+
| Qcache_free_blocks | 9 |
| Qcache_free_memory | 22736 |
| Qcache_hits | 121910 |
| Qcache_inserts | 9296 |
| Qcache_lowmem_prunes | 8213 |
| Qcache_not_cached | 29475 |
| Qcache_queries_in_cache | 103 |
| Qcache_total_blocks | 295 |
+-------------------------+--------+
正如我在上面的评论中提到的，您似乎正在从多个 table 关系中获取大量数据。这会导致几个 Cartesian products，这就是您获得 626K 行的原因。
解决方案是将其分解为单独的查询。
以下是我对您的模型的了解：
这是各个查询的样子，以及它们各自的 EXPLAIN 计划。
您可以看到这消除了所有子查询和临时 tables，它消除了笛卡尔积，因此您不需要在任何地方使用 DISTINCT，并且所有连接都很好地针对主键。它可能看起来是重复的，但查询得到更好的优化这一事实应该会使它 运行 更快。
EXPLAIN
SELECT r.res_id, r.res_status, r.res_type, r.res_category, r.res_title, r.res_filename,
r.res_source, r.res_gradelevel_from, r.res_gradelevel_to, r.res_tags, r.res_description,
r.res_onserver, r.res_favoriteaccts, r.res_CreatedOn, c.c_name AS 'Categories'
FROM hre_resources AS r
INNER JOIN categories c ON c_id = r.res_category
WHERE r.res_status = 'Active'
--------------
+----+-------------+-------+--------+-------------------------+------------+---------+---------------------+------+-----------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-------------------------+------------+---------+---------------------+------+-----------------------+
| 1 | SIMPLE | r | ref | res_status,res_category | res_status | 1 | const | 1 | Using index condition |
| 1 | SIMPLE | c | eq_ref | PRIMARY | PRIMARY | 4 | test.r.res_category | 1 | NULL |
+----+-------------+-------+--------+-------------------------+------------+---------+---------------------+------+-----------------------+
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(s.s_name ORDER BY s.s_id SEPARATOR ', ') AS 'SchoolSubjects'
FROM hre_resources AS r
INNER JOIN schoolsubjects_to_resource str ON str.sr_resid = r.res_id
INNER JOIN schoolsubjects s ON s.s_id = str.sr_sid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+-----------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+-----------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | str | ref | PRIMARY,sr_sid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | s | eq_ref | PRIMARY | PRIMARY | 4 | test.str.sr_sid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+-----------------+------+--------------------------+
3 rows in set (0.00 sec)
--------------
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(t.t_name ORDER BY t_id SEPARATOR ', ') AS 'Topics'
FROM hre_resources AS r
INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
INNER JOIN topics t on t.t_id = ttr.rt_topicid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+---------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+---------------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | ttr | ref | PRIMARY,rt_topicid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | t | eq_ref | PRIMARY | PRIMARY | 4 | test.ttr.rt_topicid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+---------------------+------+--------------------------+
3 rows in set (0.00 sec)
--------------
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(DISTINCT CONCAT(ch.ch_code, ':', ch.ch_title) ORDER BY ch.ch_id SEPARATOR ', ') AS 'Chapters'
FROM hre_resources AS r
INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
INNER JOIN topics t on t.t_id = ttr.rt_topicid
INNER JOIN topic_to_framework ttc on ttc.ft_topicid = t.t_id
INNER JOIN ca_hss_frameworkchapters ch ON ch.ch_id = ttc.ft_frameworkid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+-------------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+-------------------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | ttr | ref | PRIMARY,rt_topicid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | t | eq_ref | PRIMARY | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | ttc | ref | PRIMARY,ft_frameworkid | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | ch | eq_ref | PRIMARY | PRIMARY | 4 | test.ttc.ft_frameworkid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+-------------------------+------+--------------------------+
5 rows in set (0.00 sec)
--------------
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(DISTINCT cs.cs_code ORDER BY cs.cs_id SEPARATOR ', ') AS 'ContentStandards'
FROM hre_resources AS r
INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
INNER JOIN topics t on t.t_id = ttr.rt_topicid
INNER JOIN topic_to_cs tcs on tcs.cst_topicid = t.t_id
INNER JOIN ca_hss_contentstandards cs ON cs.cs_id = tcs.cst_contentstandardid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+--------------------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+--------------------------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | ttr | ref | PRIMARY,rt_topicid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | t | eq_ref | PRIMARY | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | tcs | ref | PRIMARY,cst_contentstandardid | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | cs | eq_ref | PRIMARY | PRIMARY | 4 | test.tcs.cst_contentstandardid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+--------------------------------+------+--------------------------+
下面是我为 table 假定的 DDL。如果您将其包含在您的原始问题中，那将会很有帮助。
CREATE TABLE categories (
c_id INT PRIMARY KEY,
c_name VARCHAR(100) NOT NULL DEFAULT 'c_name'
);
CREATE TABLE hre_resources (
res_id INT PRIMARY KEY,
res_category INT NOT NULL,
res_status ENUM('Active', 'Inactive') NOT NULL DEFAULT 'Active',
res_type VARCHAR(100) NOT NULL DEFAULT 'type',
res_title VARCHAR(100) NOT NULL DEFAULT 'title',
res_filename VARCHAR(100) NOT NULL DEFAULT '',
res_source VARCHAR(100) NOT NULL DEFAULT '',
res_gradelevel_from VARCHAR(100) NOT NULL DEFAULT '',
res_gradelevel_to VARCHAR(100) NOT NULL DEFAULT '',
res_tags VARCHAR(100) NOT NULL DEFAULT '',
res_description VARCHAR(100) NOT NULL DEFAULT '',
res_onserver VARCHAR(100) NOT NULL DEFAULT '',
res_favoriteaccts VARCHAR(100) NOT NULL DEFAULT '',
res_CreatedOn DATE NOT NULL DEFAULT '2017-01-01',
KEY (res_status),
FOREIGN KEY (res_category) REFERENCES categories(c_id)
);
CREATE TABLE topics (
t_id INT PRIMARY KEY,
t_name VARCHAR(100) NOT NULL DEFAULT 't_name'
);
CREATE TABLE ca_hss_frameworkchapters (
ch_id INT PRIMARY KEY,
ch_title VARCHAR(100) NOT NULL DEFAULT 'ch_title',
ch_code VARCHAR(100) NOT NULL DEFAULT 'ch_code'
);
CREATE TABLE ca_hss_contentstandards (
cs_id INT PRIMARY KEY,
cs_code VARCHAR(100) NOT NULL DEFAULT 'cs_code'
);
CREATE TABLE schoolsubjects (
s_id INT PRIMARY KEY,
s_name VARCHAR(100) NOT NULL DEFAULT 's_name'
);
CREATE TABLE topic_to_framework (
ft_topicid INT NOT NULL,
ft_frameworkid INT NOT NULL,
PRIMARY KEY (ft_topicid, ft_frameworkid),
FOREIGN KEY (ft_topicid) REFERENCES topics(t_id),
FOREIGN KEY (ft_frameworkid) REFERENCES ca_hss_frameworkchapters(ch_id)
);
CREATE TABLE topic_to_resource (
rt_resourceid INT NOT NULL,
rt_topicid INT NOT NULL,
PRIMARY KEY (rt_resourceid, rt_topicid),
FOREIGN KEY (rt_resourceid) REFERENCES hre_resources (res_id),
FOREIGN KEY (rt_topicid) REFERENCES topics(t_id)
);
CREATE TABLE topic_to_cs (
cst_topicid INT NOT NULL,
cst_contentstandardid INT NOT NULL,
PRIMARY KEY (cst_topicid, cst_contentstandardid),
FOREIGN KEY (cst_topicid) REFERENCES topics(t_id),
FOREIGN KEY (cst_contentstandardid) REFERENCES ca_hss_contentstandards(cs_id)
);
CREATE TABLE schoolsubjects_to_resource (
sr_resid INT NOT NULL,
sr_sid INT NOT NULL,
PRIMARY KEY (sr_resid, sr_sid),
FOREIGN KEY (sr_resid) REFERENCES hre_resources(res_id),
FOREIGN KEY (sr_sid) REFERENCES schoolsubjects(s_id)
);
下面是 Python 中的一个示例，说明了我将多个查询的结果合并在一起的意思。
import pprint
import MySQLdb
import MySQLdb.cursors
db = MySQLdb.connect(host="localhost", db="test", cursorclass=MySQLdb.cursors.DictCursor)
cur = db.cursor()
res_data = {}
sql = """SELECT r.res_id, r.res_status, r.res_type, r.res_category, r.res_title, r.res_filename, r.res_source, r.res_gradelevel_from, r.res_gradelevel_to, r.res_tags, r.res_description, r.res_onserver, r.res_favoriteaccts, r.res_CreatedOn, c.c_name AS 'Categories' FROM hre_resources AS r INNER JOIN categories c ON c_id = r.res_category WHERE r.res_status = 'Active'"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']] = row
sql = """SELECT r.res_id, GROUP_CONCAT(s.s_name ORDER BY s.s_id SEPARATOR ', ') AS 'SchoolSubjects' FROM hre_resources AS r INNER JOIN schoolsubjects_to_resource str ON str.sr_resid = r.res_id INNER JOIN schoolsubjects s ON s.s_id = str.sr_sid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
sql = """SELECT r.res_id, GROUP_CONCAT(t.t_name ORDER BY t_id SEPARATOR ', ') AS 'Topics' FROM hre_resources AS r INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id INNER JOIN topics t on t.t_id = ttr.rt_topicid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
sql = """SELECT r.res_id, GROUP_CONCAT(DISTINCT CONCAT(ch.ch_code, ':', ch.ch_title) ORDER BY ch.ch_id SEPARATOR ', ') AS 'Chapters' FROM hre_resources AS r INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id INNER JOIN topics t on t.t_id = ttr.rt_topicid INNER JOIN topic_to_framework ttc on ttc.ft_topicid = t.t_id INNER JOIN ca_hss_frameworkchapters ch ON ch.ch_id = ttc.ft_frameworkid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
sql = """SELECT r.res_id, GROUP_CONCAT(DISTINCT cs.cs_code ORDER BY cs.cs_id SEPARATOR ', ') AS 'ContentStandards' FROM hre_resources AS r INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id INNER JOIN topics t on t.t_id = ttr.rt_topicid INNER JOIN topic_to_cs tcs on tcs.cst_topicid = t.t_id INNER JOIN ca_hss_contentstandards cs ON cs.cs_id = tcs.cst_contentstandardid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
db.close()
pp = pprint.PrettyPrinter()
pp.pprint(res_data)
在我向每个 table 添加一行假数据后，我从上面的脚本中得到了这个输出：
{2L: {'Categories': 'c_name',
'Chapters': 'ch_codech_title',
'ContentStandards': 'cs_code',
'SchoolSubjects': 's_name',
'Topics': 't_name',
'res_CreatedOn': datetime.date(2017, 1, 1),
'res_category': 1L,
'res_description': '',
'res_favoriteaccts': '',
'res_filename': '',
'res_gradelevel_from': '',
'res_gradelevel_to': '',
'res_id': 2L,
'res_onserver': '',
'res_source': '',
'res_status': 'Active',
'res_tags': '',
'res_title': 'title',
'res_type': 'type'}}
首先将 WHERE RES_Status = 'Active' 拉入派生的 table。这可能会显着减小 tmp table 的大小，并可能显着缩小 "Creating sort index" 阶段。
然后关注派生table。把它分开。需要多长时间？
至于56.7s的部分，听起来有些设置太高了。请提供：
SHOW VARIABLES LIKE 'query%';
SHOW GLOBAL STATUS LIKE 'Qc%';
我正在使用以下查询：
SELECT *,
GROUP_CONCAT(distinct t_name order by t_id separator ', ') 'Topics',
GROUP_CONCAT(distinct ch_title order by ch_id separator ', ') 'Chapters',
GROUP_CONCAT(distinct cs_code order by cs_id separator ', ') 'ContentStandards',
GROUP_CONCAT(distinct c_name order by c_id separator ', ') 'Categories',
GROUP_CONCAT(distinct s_name order by s_id separator ', ') 'SchoolSubjects'
FROM (
SELECT r.res_id, r.res_status, r.res_type, r.res_category, r.res_title, r.res_filename, r.res_source, r.res_gradelevel_from, r.res_gradelevel_to, r.res_tags, r.res_description, r.res_onserver, r.res_favoriteaccts, r.res_CreatedOn, c_id, s_id, t_id, ch_id, cs_id,
CONCAT("<a class=""ReportLink"" href=""page.asp?category=",c_name,""" title=""",c_name,""">",c_name,"</a>") as c_name,
CONCAT("<a class=""ReportLink"" href=""page.asp?subject=",s_name,""" title=""",s_name,""">",s_name,"</a>") as s_name,
CONCAT("<a class=""ReportLink"" href=""page.asp?topic=",t_name,""" title=""",t_name,""">",t_name,"</a>") as t_name,
CONCAT("<a class=""ReportLink"" href=""page.asp?cs=",cs_code,""" title=""",cs_code,""">",cs_code,"</a>") as cs_code,
CONCAT("<a class=""ReportLink"" href=""page.asp?chapter=",ch_title,""" title=""",ch_title,""">",ch_code,": ",ch_title,"</a>") as ch_title
FROM hre_resources r
LEFT JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
LEFT JOIN topics t on t.t_id = ttr.rt_topicid
LEFT JOIN topic_to_cs tcs on tcs.cst_topicid = t.t_id
LEFT JOIN categories c ON c_id = r.res_category
LEFT JOIN topic_to_framework ttc on ttc.ft_topicid = t.t_id
LEFT JOIN ca_hss_frameworkchapters ch ON ch.ch_id = ttc.ft_frameworkid
LEFT JOIN ca_hss_contentstandards cs ON cs.cs_id = tcs.cst_contentstandardid
LEFT JOIN schoolsubjects_to_resource str ON str.sr_resid = r.res_id
LEFT JOIN schoolsubjects s ON s.s_id = str.sr_sid) links
WHERE RES_Status = 'Active'
GROUP BY RES_ID
ORDER BY RES_Title ASC, RES_Source DESC
此查询 returns 来自我们 mySQL 数据库的 64 条格式完美的记录，但执行此操作需要 100 多秒。
查询分析器将主要违规者显示为：
+-------------------------------+-----------+
| Sending data | 34.530026 |
+-------------------------------+-----------+
| Creating sort index | 24.205878 |
| Storing result in query cache | 56.739204 |
+-------------------------------+-----------+
解释结果如下：
+----+-------------+------------+--------+---------------+---------------+---------+------------------------------------+------+---------------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+---------------+---------------+---------+------------------------------------+------+---------------------------------------------------------------------+
| 1 | PRIMARY | <derived2> | ref | <auto_key0> | <auto_key0> | 77 | const | 10 | Using index condition; Using where; Using temporary; Using filesort |
| 2 | DERIVED | r | ALL | NULL | NULL | NULL | NULL | 64 | NULL |
| 2 | DERIVED | ttr | ref | RT_ResourceID | RT_ResourceID | 4 | uhri_hre.r.RES_ID | 3 | Using where |
| 2 | DERIVED | t | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.ttr.RT_TopicID | 1 | Using where |
| 2 | DERIVED | tcs | ref | CST_TopicID | CST_TopicID | 4 | uhri_hre.t.T_ID | 9 | Using where |
| 2 | DERIVED | c | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.r.RES_Category | 1 | Using where |
| 2 | DERIVED | ttc | ref | FT_TopicID | FT_TopicID | 4 | uhri_hre.t.T_ID | 21 | Using where |
| 2 | DERIVED | ch | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.ttc.FT_FrameworkID | 1 | Using where |
| 2 | DERIVED | cs | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.tcs.CST_ContentStandardID | 1 | Using where |
| 2 | DERIVED | str | ref | SR_RESID | SR_RESID | 4 | uhri_hre.r.RES_ID | 1 | Using where |
| 2 | DERIVED | s | eq_ref | PRIMARY | PRIMARY | 4 | uhri_hre.str.SR_SID | 1 | Using where |
+----+-------------+------------+--------+---------------+---------------+---------+------------------------------------+------+---------------------------------------------------------------------+
这些表具有以下索引。未列出的表在 ID 字段（主键）上有索引：
HRE_Resources:
+--------------+-------+--------+--------+--------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+--------------+-------+--------+--------+--------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | RES_ID | 64 | A | | |
| RES_Type | BTREE | No | No | RES_Type | 4 | A | | |
| RES_Category | BTREE | No | No | RES_Category | 10 | A | | |
| RES_Status | BTREE | No | No | RES_Status | 4 | A | | |
| RES_Language | BTREE | No | No | RES_Language | 8 | A | | |
| RES_Source | BTREE | No | No | RES_Source | 21 | A | | |
+--------------+-------+--------+--------+--------------+-------------+-----------+------+---------+
Topics:
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | T_ID | 39 | A | | |
| T_Name | BTREE | No | No | T_Name | 39 | A | | |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
Topic_to_CS:
+-----------------------+-------+--------+--------+-----------------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+-----------------------+-------+--------+--------+-----------------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | CST_ID | 685 | A | | |
| CST_ContentStandardID | BTREE | No | No | CST_ContentStandardID | 228 | A | | |
| CST_TopicID | BTREE | No | No | CST_TopicID | 76 | A | | |
+-----------------------+-------+--------+--------+-----------------------+-------------+-----------+------+---------+
Topic_to_Framework:
+----------------+-------+--------+--------+----------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+----------------+-------+--------+--------+----------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | FT_ID | 471 | A | | |
| FT_TopicID | BTREE | No | No | FT_TopicID | 22 | A | | |
| FT_FrameworkID | BTREE | No | No | FT_FrameworkID | 235 | A | | |
+----------------+-------+--------+--------+----------------+-------------+-----------+------+---------+
Topic_to_Resource:
+---------------+-------+--------+--------+---------------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+---------------+-------+--------+--------+---------------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | RT_ID | 387 | A | | |
| RT_ResourceID | BTREE | No | No | RT_ResourceID | 129 | A | | |
| RT_TopicID | BTREE | No | No | RT_TopicID | 77 | A | | |
+---------------+-------+--------+--------+---------------+-------------+-----------+------+---------+
Schoolsubject_to_Resource:
+----------+-------+--------+--------+----------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+----------+-------+--------+--------+----------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | SR_ID | 228 | A | | |
| SR_SID | BTREE | No | No | SR_SID | 25 | A | | |
| SR_RESID | BTREE | No | No | SR_RESID | 228 | A | | |
+----------+-------+--------+--------+----------+-------------+-----------+------+---------+
Categories:
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| Keyname | Type | Unique | Packed | Column | Cardinality | Collation | Null | Comment |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
| PRIMARY | BTREE | Yes | No | C_ID | 9 | A | | |
| C_Name | BTREE | No | No | C_Name | 9 | A | | |
+---------+-------+--------+--------+--------+-------------+-----------+------+---------+
其中一些索引是使用其他查询中使用的更复杂的 WHERE 创建的。
出于其他目的，我对数据库进行了 运行 复杂查询，这些查询与上述查询无关，而且它的执行名义上是正常的。因此，我认为这不是服务器资源问题。
任何人都可以在上面的查询中看到任何可能导致响应如此缓慢的内容吗？
非常感谢！
PS。根据以下请求从服务器添加一些信息...
+------------------------------+---------+
| Variable_name | Value |
+------------------------------+---------+
| query_alloc_block_size | 8192 |
| query_cache_limit | 1048576 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 1048576 |
| query_cache_type | ON |
| query_cache_wlock_invalidate | OFF |
| query_prealloc_size | 8192 |
+------------------------------+---------+
+-------------------------+--------+
| Variable_name | Value |
+-------------------------+--------+
| Qcache_free_blocks | 9 |
| Qcache_free_memory | 22736 |
| Qcache_hits | 121910 |
| Qcache_inserts | 9296 |
| Qcache_lowmem_prunes | 8213 |
| Qcache_not_cached | 29475 |
| Qcache_queries_in_cache | 103 |
| Qcache_total_blocks | 295 |
+-------------------------+--------+
正如我在上面的评论中提到的，您似乎正在从多个 table 关系中获取大量数据。这会导致几个 Cartesian products，这就是您获得 626K 行的原因。
解决方案是将其分解为单独的查询。
以下是我对您的模型的了解：
这是各个查询的样子，以及它们各自的 EXPLAIN 计划。
您可以看到这消除了所有子查询和临时 tables，它消除了笛卡尔积，因此您不需要在任何地方使用 DISTINCT，并且所有连接都很好地针对主键。它可能看起来是重复的，但查询得到更好的优化这一事实应该会使它 运行 更快。
EXPLAIN
SELECT r.res_id, r.res_status, r.res_type, r.res_category, r.res_title, r.res_filename,
r.res_source, r.res_gradelevel_from, r.res_gradelevel_to, r.res_tags, r.res_description,
r.res_onserver, r.res_favoriteaccts, r.res_CreatedOn, c.c_name AS 'Categories'
FROM hre_resources AS r
INNER JOIN categories c ON c_id = r.res_category
WHERE r.res_status = 'Active'
--------------
+----+-------------+-------+--------+-------------------------+------------+---------+---------------------+------+-----------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+-------------------------+------------+---------+---------------------+------+-----------------------+
| 1 | SIMPLE | r | ref | res_status,res_category | res_status | 1 | const | 1 | Using index condition |
| 1 | SIMPLE | c | eq_ref | PRIMARY | PRIMARY | 4 | test.r.res_category | 1 | NULL |
+----+-------------+-------+--------+-------------------------+------------+---------+---------------------+------+-----------------------+
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(s.s_name ORDER BY s.s_id SEPARATOR ', ') AS 'SchoolSubjects'
FROM hre_resources AS r
INNER JOIN schoolsubjects_to_resource str ON str.sr_resid = r.res_id
INNER JOIN schoolsubjects s ON s.s_id = str.sr_sid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+-----------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+-----------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | str | ref | PRIMARY,sr_sid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | s | eq_ref | PRIMARY | PRIMARY | 4 | test.str.sr_sid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+-----------------+------+--------------------------+
3 rows in set (0.00 sec)
--------------
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(t.t_name ORDER BY t_id SEPARATOR ', ') AS 'Topics'
FROM hre_resources AS r
INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
INNER JOIN topics t on t.t_id = ttr.rt_topicid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+---------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+---------------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | ttr | ref | PRIMARY,rt_topicid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | t | eq_ref | PRIMARY | PRIMARY | 4 | test.ttr.rt_topicid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+---------------------+------+--------------------------+
3 rows in set (0.00 sec)
--------------
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(DISTINCT CONCAT(ch.ch_code, ':', ch.ch_title) ORDER BY ch.ch_id SEPARATOR ', ') AS 'Chapters'
FROM hre_resources AS r
INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
INNER JOIN topics t on t.t_id = ttr.rt_topicid
INNER JOIN topic_to_framework ttc on ttc.ft_topicid = t.t_id
INNER JOIN ca_hss_frameworkchapters ch ON ch.ch_id = ttc.ft_frameworkid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+-------------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+-------------------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | ttr | ref | PRIMARY,rt_topicid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | t | eq_ref | PRIMARY | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | ttc | ref | PRIMARY,ft_frameworkid | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | ch | eq_ref | PRIMARY | PRIMARY | 4 | test.ttc.ft_frameworkid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+-------------------------+------+--------------------------+
5 rows in set (0.00 sec)
--------------
EXPLAIN
SELECT r.res_id, GROUP_CONCAT(DISTINCT cs.cs_code ORDER BY cs.cs_id SEPARATOR ', ') AS 'ContentStandards'
FROM hre_resources AS r
INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id
INNER JOIN topics t on t.t_id = ttr.rt_topicid
INNER JOIN topic_to_cs tcs on tcs.cst_topicid = t.t_id
INNER JOIN ca_hss_contentstandards cs ON cs.cs_id = tcs.cst_contentstandardid
WHERE r.res_status = 'Active'
GROUP BY r.res_id
--------------
+----+-------------+-------+--------+---------------------------------+------------+---------+--------------------------------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------------------------+------------+---------+--------------------------------+------+--------------------------+
| 1 | SIMPLE | r | ref | PRIMARY,res_status,res_category | res_status | 1 | const | 1 | Using where; Using index |
| 1 | SIMPLE | ttr | ref | PRIMARY,rt_topicid | PRIMARY | 4 | test.r.res_id | 1 | Using index |
| 1 | SIMPLE | t | eq_ref | PRIMARY | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | tcs | ref | PRIMARY,cst_contentstandardid | PRIMARY | 4 | test.ttr.rt_topicid | 1 | Using index |
| 1 | SIMPLE | cs | eq_ref | PRIMARY | PRIMARY | 4 | test.tcs.cst_contentstandardid | 1 | NULL |
+----+-------------+-------+--------+---------------------------------+------------+---------+--------------------------------+------+--------------------------+
下面是我为 table 假定的 DDL。如果您将其包含在您的原始问题中，那将会很有帮助。
CREATE TABLE categories (
c_id INT PRIMARY KEY,
c_name VARCHAR(100) NOT NULL DEFAULT 'c_name'
);
CREATE TABLE hre_resources (
res_id INT PRIMARY KEY,
res_category INT NOT NULL,
res_status ENUM('Active', 'Inactive') NOT NULL DEFAULT 'Active',
res_type VARCHAR(100) NOT NULL DEFAULT 'type',
res_title VARCHAR(100) NOT NULL DEFAULT 'title',
res_filename VARCHAR(100) NOT NULL DEFAULT '',
res_source VARCHAR(100) NOT NULL DEFAULT '',
res_gradelevel_from VARCHAR(100) NOT NULL DEFAULT '',
res_gradelevel_to VARCHAR(100) NOT NULL DEFAULT '',
res_tags VARCHAR(100) NOT NULL DEFAULT '',
res_description VARCHAR(100) NOT NULL DEFAULT '',
res_onserver VARCHAR(100) NOT NULL DEFAULT '',
res_favoriteaccts VARCHAR(100) NOT NULL DEFAULT '',
res_CreatedOn DATE NOT NULL DEFAULT '2017-01-01',
KEY (res_status),
FOREIGN KEY (res_category) REFERENCES categories(c_id)
);
CREATE TABLE topics (
t_id INT PRIMARY KEY,
t_name VARCHAR(100) NOT NULL DEFAULT 't_name'
);
CREATE TABLE ca_hss_frameworkchapters (
ch_id INT PRIMARY KEY,
ch_title VARCHAR(100) NOT NULL DEFAULT 'ch_title',
ch_code VARCHAR(100) NOT NULL DEFAULT 'ch_code'
);
CREATE TABLE ca_hss_contentstandards (
cs_id INT PRIMARY KEY,
cs_code VARCHAR(100) NOT NULL DEFAULT 'cs_code'
);
CREATE TABLE schoolsubjects (
s_id INT PRIMARY KEY,
s_name VARCHAR(100) NOT NULL DEFAULT 's_name'
);
CREATE TABLE topic_to_framework (
ft_topicid INT NOT NULL,
ft_frameworkid INT NOT NULL,
PRIMARY KEY (ft_topicid, ft_frameworkid),
FOREIGN KEY (ft_topicid) REFERENCES topics(t_id),
FOREIGN KEY (ft_frameworkid) REFERENCES ca_hss_frameworkchapters(ch_id)
);
CREATE TABLE topic_to_resource (
rt_resourceid INT NOT NULL,
rt_topicid INT NOT NULL,
PRIMARY KEY (rt_resourceid, rt_topicid),
FOREIGN KEY (rt_resourceid) REFERENCES hre_resources (res_id),
FOREIGN KEY (rt_topicid) REFERENCES topics(t_id)
);
CREATE TABLE topic_to_cs (
cst_topicid INT NOT NULL,
cst_contentstandardid INT NOT NULL,
PRIMARY KEY (cst_topicid, cst_contentstandardid),
FOREIGN KEY (cst_topicid) REFERENCES topics(t_id),
FOREIGN KEY (cst_contentstandardid) REFERENCES ca_hss_contentstandards(cs_id)
);
CREATE TABLE schoolsubjects_to_resource (
sr_resid INT NOT NULL,
sr_sid INT NOT NULL,
PRIMARY KEY (sr_resid, sr_sid),
FOREIGN KEY (sr_resid) REFERENCES hre_resources(res_id),
FOREIGN KEY (sr_sid) REFERENCES schoolsubjects(s_id)
);
下面是 Python 中的一个示例，说明了我将多个查询的结果合并在一起的意思。
import pprint
import MySQLdb
import MySQLdb.cursors
db = MySQLdb.connect(host="localhost", db="test", cursorclass=MySQLdb.cursors.DictCursor)
cur = db.cursor()
res_data = {}
sql = """SELECT r.res_id, r.res_status, r.res_type, r.res_category, r.res_title, r.res_filename, r.res_source, r.res_gradelevel_from, r.res_gradelevel_to, r.res_tags, r.res_description, r.res_onserver, r.res_favoriteaccts, r.res_CreatedOn, c.c_name AS 'Categories' FROM hre_resources AS r INNER JOIN categories c ON c_id = r.res_category WHERE r.res_status = 'Active'"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']] = row
sql = """SELECT r.res_id, GROUP_CONCAT(s.s_name ORDER BY s.s_id SEPARATOR ', ') AS 'SchoolSubjects' FROM hre_resources AS r INNER JOIN schoolsubjects_to_resource str ON str.sr_resid = r.res_id INNER JOIN schoolsubjects s ON s.s_id = str.sr_sid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
sql = """SELECT r.res_id, GROUP_CONCAT(t.t_name ORDER BY t_id SEPARATOR ', ') AS 'Topics' FROM hre_resources AS r INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id INNER JOIN topics t on t.t_id = ttr.rt_topicid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
sql = """SELECT r.res_id, GROUP_CONCAT(DISTINCT CONCAT(ch.ch_code, ':', ch.ch_title) ORDER BY ch.ch_id SEPARATOR ', ') AS 'Chapters' FROM hre_resources AS r INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id INNER JOIN topics t on t.t_id = ttr.rt_topicid INNER JOIN topic_to_framework ttc on ttc.ft_topicid = t.t_id INNER JOIN ca_hss_frameworkchapters ch ON ch.ch_id = ttc.ft_frameworkid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
sql = """SELECT r.res_id, GROUP_CONCAT(DISTINCT cs.cs_code ORDER BY cs.cs_id SEPARATOR ', ') AS 'ContentStandards' FROM hre_resources AS r INNER JOIN topic_to_resource ttr ON ttr.rt_resourceid = r.res_id INNER JOIN topics t on t.t_id = ttr.rt_topicid INNER JOIN topic_to_cs tcs on tcs.cst_topicid = t.t_id INNER JOIN ca_hss_contentstandards cs ON cs.cs_id = tcs.cst_contentstandardid WHERE r.res_status = 'Active' GROUP BY r.res_id"""
cur.execute(sql)
for row in cur.fetchall():
res_data[row['res_id']].update(row)
db.close()
pp = pprint.PrettyPrinter()
pp.pprint(res_data)
在我向每个 table 添加一行假数据后，我从上面的脚本中得到了这个输出：
{2L: {'Categories': 'c_name',
'Chapters': 'ch_codech_title',
'ContentStandards': 'cs_code',
'SchoolSubjects': 's_name',
'Topics': 't_name',
'res_CreatedOn': datetime.date(2017, 1, 1),
'res_category': 1L,
'res_description': '',
'res_favoriteaccts': '',
'res_filename': '',
'res_gradelevel_from': '',
'res_gradelevel_to': '',
'res_id': 2L,
'res_onserver': '',
'res_source': '',
'res_status': 'Active',
'res_tags': '',
'res_title': 'title',
'res_type': 'type'}}
首先将 WHERE RES_Status = 'Active' 拉入派生的 table。这可能会显着减小 tmp table 的大小，并可能显着缩小 "Creating sort index" 阶段。
然后关注派生table。把它分开。需要多长时间？
至于56.7s的部分，听起来有些设置太高了。请提供：
SHOW VARIABLES LIKE 'query%';
SHOW GLOBAL STATUS LIKE 'Qc%';