减少 ClickHouse 中的资源消耗

Question

table

CREATE TABLE events
(
    site_id UInt64,
    name    String
    -- other columns
)
ENGINE = CollapsingMergeTree(sign_flag)
PARTITION BY site_id
ORDER BY (name)
SETTINGS index_granularity = 8192;

查询

SELECT 'wtf',
       *
FROM events
WHERE site_id = 1 AND
      name = 'some_name'
LIMIT 100000;

日志

SELECT formatReadableSize(read_bytes)    AS read_bytes,
       formatReadableSize(memory_usage)  AS memory_usage,
       formatReadableQuantity(read_rows) AS read_rows,
       query_duration_ms / 1000 AS query_duration_sec,
       query
FROM system.query_log
WHERE query LIKE '%wtf%'
ORDER BY
    event_time DESC
LIMIT 100;

+------------+--------------+--------------+--------------------+
| read_bytes | memory_usage | read_rows    | query_duration_sec |
+------------+--------------+--------------+--------------------+
| 578.41 MiB | 131.95 MiB   | 1.01 million | 10.773             |
+------------+--------------+--------------+--------------------+

我认为日志中的数字非常多。

如何优化它或者我错过了一些关于服务器配置的东西？

Answer 1

考虑定义另一个主键 - 对于此查询 ORDER BY (name, site_id).

选择PK是设计中非常重要的一环，要选择合适的PK需要观察所有的用例图。详情请见：

ClickHouse: Selecting the Primary Key
Whosebug #62556274.

减少 ClickHouse 中的资源消耗

Reduce resource consumption in ClickHouse

clickhouse