优化疯狂 MySQL 查询
Optimize crazy MySQL query
我需要优化这个看起来很疯狂的查询（遗留代码）：
SELECT
E.eventId,
E.currency,
COALESCE(ROUND(UR.ratings, 2), 0) as ratings,
COALESCE(UR.ratingCount, 0) as ratingCount,
E.shopSpaceAvail,
E.floorPlanImage,
COALESCE(O.goingCount, 0) as goingCount,
(COALESCE(O.goingGroup, '')) as goingGroup,
E.userId,
E.name,
E.withoutTicket,
E.mainImage,
E.mainImageThumb,
E.privateEvent,
E.location,
E.locationLatitude,
E.locationLongitude,
E.country3Code,
E.country,
E.city,
E.description,
E.startDt as startDt_formatted,
E.endDt as endDt_formatted,
(
SELECT
COUNT(*)
FROM
eventIntresteds
WHERE
eventId = E.eventId
) as interestedCount,(
CASE WHEN "kaka" = "" THEN 0 ELSE (
SELECT
COUNT(*)
FROM
eventIntresteds as EI3
WHERE
EI3.eventId = E.eventId
AND EI3.userId IN (
48,
1872,
2039,
67132,
1076,
1880,
3504,
3641,
4575,
3080,
67129,
67130,
67134
)
) END
) as mutualInterestedCount,
COALESCE(
(
SELECT
MIN(adultPrice)
FROM
eventTickets as ET
WHERE
ET.deleted = '0'
AND ET.eventId = E.eventId
AND ET.eventTicketType = 'Normal'
),
0
) as minPrice,
(
CASE WHEN 'kaka' = '' THEN '2' WHEN (
(
SELECT
COUNT(*)
FROM
eventIntresteds EI1
WHERE
EI1.eventId = E.eventId
AND EI1.userId = 2162
) > 0
) THEN '1' ELSE '0' END
) as isInterested,
(
ROUND(
(
(
3959 * acos(
cos(radians(0)) * cos(radians(E.locationLatitude)) * cos(radians(E.locationLongitude) - radians(0)) + sin(radians(0)) * sin(radians(E.locationLatitude))
)
) * 1.67
),
4
)
) as distance,
(
CASE WHEN E.privateEvent = '0'
OR E.userId = 2162 THEN '1' WHEN 'kaka' = '' THEN '0' WHEN (
(
SELECT
COUNT(*)
FROM
userNotifications UN
WHERE
UN.eventId = E.eventId
AND UN.userId = 2162
AND UN.notificationType = 'eventInvite'
) = 0
) THEN '0' ELSE '1' END
) as isprivateEvent,
(
CASE WHEN (
E.privateEvent = '0'
or E.userId = 2162
) THEN 1 ELSE (
SELECT
COUNT(*)
FROM
invites AS I
WHERE
I.eventId = E.eventId
AND I.inviteType = 'Event'
AND I.deleted = '0'
AND I.userId = 2162
) END
) as privateHavingCheck,
(
SELECT
COUNT(Distinct O1.shopId)
FROM
orders O1
JOIN shops S1 ON (S1.shopId = O1.shopId)
WHERE
O1.eventId = E.eventId
AND S1.deleted = '0'
AND S1.blocked = '0'
AND O1.orderStatus = 'Success'
AND O1.paymentStatus = 'Success'
AND O1.orderType = 'shopBooth'
AND O1.refundId = ''
) as shopCount
FROM
events AS E
JOIN users U ON (
U.userId = E.userId
AND U.blocked = '0'
)
LEFT JOIN (
SELECT
COUNT(*) as ratingCount,
AVG(ratings) as ratings,
eventId
FROM
userRatings
WHERE
userRatings.blocked = '0'
AND userRatings.deleted = '0'
GROUP BY
eventId
) as UR ON (UR.eventId = E.eventId)
LEFT JOIN (
SELECT
COUNT(*) as goingCount,
GROUP_CONCAT(userId) as goingGroup,
eventId
FROM
orders AS O
WHERE
O.orderStatus = 'Success'
AND O.paymentStatus = 'Success'
AND orderType = 'EventTicket'
GROUP BY
eventId
) as O ON (O.eventId = E.eventId)
WHERE
E.blocked = '0'
AND E.deleted = '0'
AND E.approved = '1'
AND E.privateEvent = '0'
AND E.endDt >= now()
Having
eventId != 0
AND isprivateEvent = '1'
AND distance <= 1000000
ORDER BY
minPrice DESC,
minPrice DESC
LIMIT
0, 10;
我试图为此创建索引，但似乎无济于事。查询超级慢。麦汁部分似乎是这样的：
eventId FROM orders AS O WHERE O.orderStatus='Success' AND O.paymentStatus='Success' AND orderType='EventTicket' GROUP BY eventId) as O ON (O.eventId=E.eventId)
这根据 explain() 扫描了 10K+ 行。我在 (orderStatus,paymentStatus,orderType) 上尝试了复合索引，但它没有帮助。
关于如何快速优化此查询有什么建议吗？我知道它应该重构，但没有时间这样做。我也知道它不好，所以我不希望它超快。但在这一点上，任何加速都将不胜感激。
这是MySQL5.7.
编辑：
解释如下：
+----+--------------------+-----------------+------------+-------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------+----------+----------------------------------------+-------+----------+-----------------------------------------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+-----------------+------------+-------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------+----------+----------------------------------------+-------+----------+-----------------------------------------------------------------------------------+
| 1 | PRIMARY | E | NULL | ref | userId,combo,events_idx_blocke_delete_approv_privat_enddt | events_idx_blocke_delete_approv_privat_enddt | 4 | const,const,const,const | 359 | 33.33 | Using index condition; Using where; Using temporary; Using filesort |
| 1 | PRIMARY | U | NULL | eq_ref | PRIMARY,blocked,users_idx_blocked_userid | PRIMARY | 8 | db.E.userId | 1 | 50.00 | Using where |
| 1 | PRIMARY | <derived9> | NULL | ref | <auto_key0> | <auto_key0> | 8 | db.E.eventId | 2 | 100.00 | NULL |
| 1 | PRIMARY | <derived10> | NULL | ref | <auto_key0> | <auto_key0> | 8 | db.E.eventId | 18 | 100.00 | NULL |
| 10 | DERIVED | O | NULL | index_merge | paymentStatus,orderStatus,orderType,eventIdAndDate,eventId,complexIdx3,complexIdx4,complexIdx5,sds | paymentStatus,orderStatus,orderType | 22,22,22 | NULL | 10019 | 100.00 | Using intersect(paymentStatus,orderStatus,orderType); Using where; Using filesort |
| 9 | DERIVED | userRatings | NULL | index | complexIdx1,eventId | eventId | 8 | NULL | 43 | 25.00 | Using where |
| 8 | DEPENDENT SUBQUERY | O1 | NULL | range | shopId,paymentStatus,orderStatus,orderType,refundId,eventIdAndDate,eventId,complexIdx3,complexIdx4,complexIdx5,sds | complexIdx3 | 44 | NULL | 78 | 2.50 | Using index condition; Using where |
| 8 | DEPENDENT SUBQUERY | S1 | NULL | eq_ref | PRIMARY,shopId,blocked,deleted,shops_idx_deleted_blocked_shopid | PRIMARY | 8 | db.O1.shopId | 1 | 80.93 | Using where |
| 7 | DEPENDENT SUBQUERY | I | NULL | ref | userId,eventId,invites_idx_invitet_deleted_userid_eventid | invites_idx_invitet_deleted_userid_eventid | 39 | const,const,const,db.E.eventId | 1 | 100.00 | Using index |
| 6 | DEPENDENT SUBQUERY | UN | NULL | ref | userId,evId,notType,combo,usernotifications_idx_userid_notificati_eventid | usernotifications_idx_userid_notificati_eventid | 38 | const,const,db.E.eventId | 368 | 100.00 | Using index |
| 5 | DEPENDENT SUBQUERY | EI1 | NULL | eq_ref | eventId_2,eventId,userId,eventintresteds_idx_userid_eventid | eventId_2 | 16 |db.E.eventId,const | 1 | 100.00 | Using where; Using index |
| 4 | DEPENDENT SUBQUERY | ET | NULL | ref | eventId,type,deleted,delEvId,eventtickets_idx_deleted_eventti_eventid_adultpr | eventtickets_idx_deleted_eventti_eventid_adultpr | 91 | const,const,db.E.eventId | 24 | 100.00 | Using index |
| 3 | DEPENDENT SUBQUERY | EI3 | NULL | ref | eventId_2,eventId,userId,eventintresteds_idx_userid_eventid | eventId_2 | 8 |db.E.eventId | 108 | 0.84 | Using where; Using index |
| 2 | DEPENDENT SUBQUERY | eventIntresteds | NULL | ref | eventId_2,eventId | eventId | 8 |db.E.eventId | 107 | 100.00 | Using where; Using index |
+----+--------------------+-----------------+------------+-------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------+----------+----------------------------------------+-------+----------+-----------------------------------------------------------------------------------+
编辑 2：
如果有人可以建议一个或多个索引来加快速度，我们将不胜感激。
此查询是由 ORM (Sequilize) 生成的，因此没有简单的方法可以手动更改它。
请注意，根据您的问题，我假设没有机会更改查询语法，因为您说它是由 ORM 生成的，并且您只需要索引形式的“快速修复”。
select-列表中的所有子查询在注释中都有Using index，所以它们已经使用了覆盖索引。我怀疑你能做些什么来进一步优化这些。相关子查询很难优化，因为它们必须对结果的每一行执行一次，before HAVING 子句中的条件。
Table events as E 优化得很好，如果我能猜到索引 events_idx_blocke_delete_approv_privat_enddt 在 WHERE 子句中引用的四个列上。
Table users AS U 由 PRIMARY key 访问，因此可能无法使用索引进一步优化它。
导出的tableUR需要优化。我会添加这个索引：
ALTER TABLE userRatings ADD INDEX (blocked, deleted, eventId, ratings);
（我会把索引名称留给你，因为你知道你想使用什么命名约定。）
派生的 table O 正在对三个单独的索引进行索引交集。通常单个复合索引比依赖索引合并要好。我会添加这个索引：
ALTER TABLE orders ADD INDEX (orderStatus, paymentStatus, orderType, eventId, userId);
我知道你说你在前三列上尝试了复合索引，但添加其他两列可以帮助它作为覆盖索引。
不幸的是，ORDER BY 是按相关子查询的结果排序的，因此无法优化掉 Using filesort.
这简直就是一个怪物查询，试图同时做几件事（例如所有相关的子查询）。你做的任何快速修复都不会有太大帮助。您确实需要重构它以获得重大改进。
（请为每个 table 提供 SHOW CREATE TABLE。）
一般来说 JOIN ( SELECT ... ) 的效率低于其他技术。
暂定索引：
E: INDEX(blocked, deleted, approved, privateEvent,
endDt) -- last in this index
orders: INDEX(paymentStatus, orderStatus, orderType, -- these first
refundId, eventId, shopId)
invites: INDEX(userId, deleted, inviteType, eventId)
UN: INDEX(notificationType, eventId, userId)
eventIntresteds: INDEX(eventId, userId) -- in this order
简化和优化：
( SELECT COUNT(*)
FROM eventIntresteds EI1
WHERE EI1.eventId = E.eventId
AND EI1.userId = 2162 ) > 0 ) THEN '1' ELSE '0' END
-->
EXISTS ( SELECT 1
FROM eventIntresteds EI1
WHERE EI1.eventId = E.eventId
AND EI1.userId = 2162 )
有几个 CASE 子句，一些 使用 EXISTS 可能效果更好。
我需要优化这个看起来很疯狂的查询（遗留代码）：
SELECT
E.eventId,
E.currency,
COALESCE(ROUND(UR.ratings, 2), 0) as ratings,
COALESCE(UR.ratingCount, 0) as ratingCount,
E.shopSpaceAvail,
E.floorPlanImage,
COALESCE(O.goingCount, 0) as goingCount,
(COALESCE(O.goingGroup, '')) as goingGroup,
E.userId,
E.name,
E.withoutTicket,
E.mainImage,
E.mainImageThumb,
E.privateEvent,
E.location,
E.locationLatitude,
E.locationLongitude,
E.country3Code,
E.country,
E.city,
E.description,
E.startDt as startDt_formatted,
E.endDt as endDt_formatted,
(
SELECT
COUNT(*)
FROM
eventIntresteds
WHERE
eventId = E.eventId
) as interestedCount,(
CASE WHEN "kaka" = "" THEN 0 ELSE (
SELECT
COUNT(*)
FROM
eventIntresteds as EI3
WHERE
EI3.eventId = E.eventId
AND EI3.userId IN (
48,
1872,
2039,
67132,
1076,
1880,
3504,
3641,
4575,
3080,
67129,
67130,
67134
)
) END
) as mutualInterestedCount,
COALESCE(
(
SELECT
MIN(adultPrice)
FROM
eventTickets as ET
WHERE
ET.deleted = '0'
AND ET.eventId = E.eventId
AND ET.eventTicketType = 'Normal'
),
0
) as minPrice,
(
CASE WHEN 'kaka' = '' THEN '2' WHEN (
(
SELECT
COUNT(*)
FROM
eventIntresteds EI1
WHERE
EI1.eventId = E.eventId
AND EI1.userId = 2162
) > 0
) THEN '1' ELSE '0' END
) as isInterested,
(
ROUND(
(
(
3959 * acos(
cos(radians(0)) * cos(radians(E.locationLatitude)) * cos(radians(E.locationLongitude) - radians(0)) + sin(radians(0)) * sin(radians(E.locationLatitude))
)
) * 1.67
),
4
)
) as distance,
(
CASE WHEN E.privateEvent = '0'
OR E.userId = 2162 THEN '1' WHEN 'kaka' = '' THEN '0' WHEN (
(
SELECT
COUNT(*)
FROM
userNotifications UN
WHERE
UN.eventId = E.eventId
AND UN.userId = 2162
AND UN.notificationType = 'eventInvite'
) = 0
) THEN '0' ELSE '1' END
) as isprivateEvent,
(
CASE WHEN (
E.privateEvent = '0'
or E.userId = 2162
) THEN 1 ELSE (
SELECT
COUNT(*)
FROM
invites AS I
WHERE
I.eventId = E.eventId
AND I.inviteType = 'Event'
AND I.deleted = '0'
AND I.userId = 2162
) END
) as privateHavingCheck,
(
SELECT
COUNT(Distinct O1.shopId)
FROM
orders O1
JOIN shops S1 ON (S1.shopId = O1.shopId)
WHERE
O1.eventId = E.eventId
AND S1.deleted = '0'
AND S1.blocked = '0'
AND O1.orderStatus = 'Success'
AND O1.paymentStatus = 'Success'
AND O1.orderType = 'shopBooth'
AND O1.refundId = ''
) as shopCount
FROM
events AS E
JOIN users U ON (
U.userId = E.userId
AND U.blocked = '0'
)
LEFT JOIN (
SELECT
COUNT(*) as ratingCount,
AVG(ratings) as ratings,
eventId
FROM
userRatings
WHERE
userRatings.blocked = '0'
AND userRatings.deleted = '0'
GROUP BY
eventId
) as UR ON (UR.eventId = E.eventId)
LEFT JOIN (
SELECT
COUNT(*) as goingCount,
GROUP_CONCAT(userId) as goingGroup,
eventId
FROM
orders AS O
WHERE
O.orderStatus = 'Success'
AND O.paymentStatus = 'Success'
AND orderType = 'EventTicket'
GROUP BY
eventId
) as O ON (O.eventId = E.eventId)
WHERE
E.blocked = '0'
AND E.deleted = '0'
AND E.approved = '1'
AND E.privateEvent = '0'
AND E.endDt >= now()
Having
eventId != 0
AND isprivateEvent = '1'
AND distance <= 1000000
ORDER BY
minPrice DESC,
minPrice DESC
LIMIT
0, 10;
我试图为此创建索引，但似乎无济于事。查询超级慢。麦汁部分似乎是这样的：
eventId FROM orders AS O WHERE O.orderStatus='Success' AND O.paymentStatus='Success' AND orderType='EventTicket' GROUP BY eventId) as O ON (O.eventId=E.eventId)
这根据 explain() 扫描了 10K+ 行。我在 (orderStatus,paymentStatus,orderType) 上尝试了复合索引，但它没有帮助。
关于如何快速优化此查询有什么建议吗？我知道它应该重构，但没有时间这样做。我也知道它不好，所以我不希望它超快。但在这一点上，任何加速都将不胜感激。
这是MySQL5.7.
编辑：
解释如下：
+----+--------------------+-----------------+------------+-------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------+----------+----------------------------------------+-------+----------+-----------------------------------------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+-----------------+------------+-------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------+----------+----------------------------------------+-------+----------+-----------------------------------------------------------------------------------+
| 1 | PRIMARY | E | NULL | ref | userId,combo,events_idx_blocke_delete_approv_privat_enddt | events_idx_blocke_delete_approv_privat_enddt | 4 | const,const,const,const | 359 | 33.33 | Using index condition; Using where; Using temporary; Using filesort |
| 1 | PRIMARY | U | NULL | eq_ref | PRIMARY,blocked,users_idx_blocked_userid | PRIMARY | 8 | db.E.userId | 1 | 50.00 | Using where |
| 1 | PRIMARY | <derived9> | NULL | ref | <auto_key0> | <auto_key0> | 8 | db.E.eventId | 2 | 100.00 | NULL |
| 1 | PRIMARY | <derived10> | NULL | ref | <auto_key0> | <auto_key0> | 8 | db.E.eventId | 18 | 100.00 | NULL |
| 10 | DERIVED | O | NULL | index_merge | paymentStatus,orderStatus,orderType,eventIdAndDate,eventId,complexIdx3,complexIdx4,complexIdx5,sds | paymentStatus,orderStatus,orderType | 22,22,22 | NULL | 10019 | 100.00 | Using intersect(paymentStatus,orderStatus,orderType); Using where; Using filesort |
| 9 | DERIVED | userRatings | NULL | index | complexIdx1,eventId | eventId | 8 | NULL | 43 | 25.00 | Using where |
| 8 | DEPENDENT SUBQUERY | O1 | NULL | range | shopId,paymentStatus,orderStatus,orderType,refundId,eventIdAndDate,eventId,complexIdx3,complexIdx4,complexIdx5,sds | complexIdx3 | 44 | NULL | 78 | 2.50 | Using index condition; Using where |
| 8 | DEPENDENT SUBQUERY | S1 | NULL | eq_ref | PRIMARY,shopId,blocked,deleted,shops_idx_deleted_blocked_shopid | PRIMARY | 8 | db.O1.shopId | 1 | 80.93 | Using where |
| 7 | DEPENDENT SUBQUERY | I | NULL | ref | userId,eventId,invites_idx_invitet_deleted_userid_eventid | invites_idx_invitet_deleted_userid_eventid | 39 | const,const,const,db.E.eventId | 1 | 100.00 | Using index |
| 6 | DEPENDENT SUBQUERY | UN | NULL | ref | userId,evId,notType,combo,usernotifications_idx_userid_notificati_eventid | usernotifications_idx_userid_notificati_eventid | 38 | const,const,db.E.eventId | 368 | 100.00 | Using index |
| 5 | DEPENDENT SUBQUERY | EI1 | NULL | eq_ref | eventId_2,eventId,userId,eventintresteds_idx_userid_eventid | eventId_2 | 16 |db.E.eventId,const | 1 | 100.00 | Using where; Using index |
| 4 | DEPENDENT SUBQUERY | ET | NULL | ref | eventId,type,deleted,delEvId,eventtickets_idx_deleted_eventti_eventid_adultpr | eventtickets_idx_deleted_eventti_eventid_adultpr | 91 | const,const,db.E.eventId | 24 | 100.00 | Using index |
| 3 | DEPENDENT SUBQUERY | EI3 | NULL | ref | eventId_2,eventId,userId,eventintresteds_idx_userid_eventid | eventId_2 | 8 |db.E.eventId | 108 | 0.84 | Using where; Using index |
| 2 | DEPENDENT SUBQUERY | eventIntresteds | NULL | ref | eventId_2,eventId | eventId | 8 |db.E.eventId | 107 | 100.00 | Using where; Using index |
+----+--------------------+-----------------+------------+-------------+--------------------------------------------------------------------------------------------------------------------+--------------------------------------------------+----------+----------------------------------------+-------+----------+-----------------------------------------------------------------------------------+
编辑 2：
如果有人可以建议一个或多个索引来加快速度，我们将不胜感激。 此查询是由 ORM (Sequilize) 生成的，因此没有简单的方法可以手动更改它。
请注意，根据您的问题，我假设没有机会更改查询语法，因为您说它是由 ORM 生成的，并且您只需要索引形式的“快速修复”。
select-列表中的所有子查询在注释中都有Using index，所以它们已经使用了覆盖索引。我怀疑你能做些什么来进一步优化这些。相关子查询很难优化，因为它们必须对结果的每一行执行一次，before HAVING 子句中的条件。
Table events as E 优化得很好，如果我能猜到索引 events_idx_blocke_delete_approv_privat_enddt 在 WHERE 子句中引用的四个列上。
Table users AS U 由 PRIMARY key 访问，因此可能无法使用索引进一步优化它。
导出的tableUR需要优化。我会添加这个索引：
ALTER TABLE userRatings ADD INDEX (blocked, deleted, eventId, ratings);
（我会把索引名称留给你，因为你知道你想使用什么命名约定。）
派生的 table O 正在对三个单独的索引进行索引交集。通常单个复合索引比依赖索引合并要好。我会添加这个索引：
ALTER TABLE orders ADD INDEX (orderStatus, paymentStatus, orderType, eventId, userId);
我知道你说你在前三列上尝试了复合索引，但添加其他两列可以帮助它作为覆盖索引。
不幸的是，ORDER BY 是按相关子查询的结果排序的，因此无法优化掉 Using filesort.
这简直就是一个怪物查询，试图同时做几件事（例如所有相关的子查询）。你做的任何快速修复都不会有太大帮助。您确实需要重构它以获得重大改进。
（请为每个 table 提供 SHOW CREATE TABLE。）
一般来说 JOIN ( SELECT ... ) 的效率低于其他技术。
暂定索引：
E: INDEX(blocked, deleted, approved, privateEvent,
endDt) -- last in this index
orders: INDEX(paymentStatus, orderStatus, orderType, -- these first
refundId, eventId, shopId)
invites: INDEX(userId, deleted, inviteType, eventId)
UN: INDEX(notificationType, eventId, userId)
eventIntresteds: INDEX(eventId, userId) -- in this order
简化和优化：
( SELECT COUNT(*)
FROM eventIntresteds EI1
WHERE EI1.eventId = E.eventId
AND EI1.userId = 2162 ) > 0 ) THEN '1' ELSE '0' END
-->
EXISTS ( SELECT 1
FROM eventIntresteds EI1
WHERE EI1.eventId = E.eventId
AND EI1.userId = 2162 )
有几个 CASE 子句，一些 使用 EXISTS 可能效果更好。