Postgres查询：磁盘异常I/O

Question

我的生产查询遇到了奇怪的 I/O 命中，我不知道问题出在哪里。我的意思是，这是一个简单的查询，没有任何大量连接或类似的东西...

我的 table 有大约 300K 行，并且每天增长 15/20K 行，这里是 table 定义：

CREATE TABLE public.tv_smartdevicemeasurement_modbus
(
    measurement_id integer NOT NULL DEFAULT nextval('tv_smartdevicemeasurement_modbus_measurement_id_seq'::regclass),
    insert_time timestamp with time zone NOT NULL,
    data jsonb NOT NULL,
    parent_job_id integer NOT NULL,
    smart_device_id integer NOT NULL,
    CONSTRAINT tv_smartdevicemeasurement_modbus_pkey PRIMARY KEY (measurement_id),
    CONSTRAINT tv_smartdevicemeasur_parent_job_id_1ac4609e_fk_tv_measur FOREIGN KEY (parent_job_id)
        REFERENCES public.tv_measurementjobs (job_id) MATCH SIMPLE
        ON UPDATE NO ACTION
        ON DELETE NO ACTION
        DEFERRABLE INITIALLY DEFERRED,
    CONSTRAINT tv_smartdevicemeasur_smart_device_id_62c12ed0_fk_tv_smartd FOREIGN KEY (smart_device_id)
        REFERENCES public.tv_smartdevice_modbus (device_id) MATCH SIMPLE
        ON UPDATE NO ACTION
        ON DELETE NO ACTION
        DEFERRABLE INITIALLY DEFERRED
)
WITH (
    OIDS = FALSE
)
TABLESPACE pg_default;

CREATE INDEX tv_smartdevicemeasurement_modbus_parent_job_id_1ac4609e
    ON public.tv_smartdevicemeasurement_modbus USING btree
    (parent_job_id)
    TABLESPACE pg_default;

CREATE INDEX tv_smartdevicemeasurement_modbus_smart_device_id_62c12ed0
    ON public.tv_smartdevicemeasurement_modbus USING btree
    (smart_device_id)
    TABLESPACE pg_default;

这里是查询：

EXPLAIN (ANALYZE, BUFFERS)
SELECT "tv_smartdevicemeasurement_modbus"."measurement_id",
       "tv_smartdevicemeasurement_modbus"."smart_device_id",
       "tv_smartdevicemeasurement_modbus"."parent_job_id",
       "tv_smartdevicemeasurement_modbus"."insert_time",
       "tv_smartdevicemeasurement_modbus"."data",
  (SELECT DATA->> 'VLN_AVG') AS "VLN_AVG",
  (SELECT DATA->> 'VLN3') AS "VLN3",
  (SELECT DATA->> 'VLN2') AS "VLN2",
  (SELECT DATA->> 'VLN1') AS "VLN1",
  (SELECT DATA->> 'VL1-2') AS "VL1-2",
  (SELECT DATA->> 'VL2-3') AS "VL2-3",
  (SELECT DATA->> 'VL3-1') AS "VL3-1",
  (SELECT DATA->> 'VLL_AVG') AS "VLL_AVG",
  (SELECT DATA->> 'IL_AVG') AS "IL_AVG",
  (SELECT DATA->> 'IL1') AS "IL1",
  (SELECT DATA->> 'IL2') AS "IL2",
  (SELECT DATA->> 'IL3') AS "IL3",
  (SELECT DATA->> 'PL_SUM') AS "PL_SUM",
  (SELECT DATA->> 'PL2') AS "PL2",
  (SELECT DATA->> 'PL3') AS "PL3",
  (SELECT DATA->> 'PL1') AS "PL1",
  (SELECT DATA->> 'kWh_L2') AS "kWh_L2",
  (SELECT DATA->> 'kWh_L3') AS "kWh_L3",
  (SELECT DATA->> 'kWh_L1') AS "kWh_L1",
  (SELECT DATA->> 'QL3') AS "QL3",
  (SELECT DATA->> 'QL1') AS "QL1",
  (SELECT DATA->> 'QL2') AS "QL2",
  (SELECT DATA->> 'QL_SUM') AS "QL_SUM",
  (SELECT DATA->> 'SL1') AS "SL1",
  (SELECT DATA->> 'SL2') AS "SL2",
  (SELECT DATA->> 'SL3') AS "SL3",
  (SELECT DATA->> 'SL_SUM') AS "SL_SUM",
  (SELECT DATA->> 'KVARh_L1') AS "KVARh_L1",
  (SELECT DATA->> 'KVARh_L2') AS "KVARh_L2",
  (SELECT DATA->> 'KVARh_LSUM') AS "KVARh_LSUM",
  (SELECT DATA->> 'KVARh_L3') AS "KVARh_L3",
  (SELECT DATA->> 'KVAh_L3') AS "KVAh_L3",
  (SELECT DATA->> 'KVAh_L1') AS "KVAh_L1",
  (SELECT DATA->> 'KVAh_L2') AS "KVAh_L2",
  (SELECT DATA->> 'KVAh_LSUM') AS "KVAh_LSUM",
  (SELECT DATA->> 'Cos_LSUM') AS "Cos_LSUM",
  (SELECT DATA->> 'Cos_L1') AS "Cos_L1",
  (SELECT DATA->> 'Cos_L2') AS "Cos_L2",
  (SELECT DATA->> 'Cos_L3') AS "Cos_L3"
FROM "tv_smartdevicemeasurement_modbus"
WHERE ("tv_smartdevicemeasurement_modbus"."insert_time" > '2019-02-07 12:15:00+01:00'
       AND "tv_smartdevicemeasurement_modbus"."insert_time" < '2019-02-07 12:31:00+01:00'
       AND "tv_smartdevicemeasurement_modbus"."smart_device_id" = 6)

还有带有Explain analyze buffer的执行计划：https://explain.depesz.com/s/d3f

非常非常奇怪！

Answer 1

一点也不奇怪或不正常。

由于你在insert_time上没有索引，PostgreSQL只能使用smart_device_id上的索引来加速查询。

但是那个条件选择性不是很强，它有 30270 个结果行，如果它们满足 insert_time 上的附加过滤条件，则必须检查每个结果行。此过滤器将删除其中的 30258 行，仅留下 12 行。

要从 table 中获取这 30270 行，PostgreSQL 必须从磁盘读取 21442 个块，每个块大小为 8 KB，这就是您花费时间的地方。

最佳索引为：

CREATE INDEX ON tv_smartdevicemeasurement_modbus (smart_device_id, insert_time);

但我怀疑以下更广泛适用且更小的索引也会提供良好的响应时间：

CREATE INDEX ON tv_smartdevicemeasurement_modbus (insert_time);

Postgres查询：磁盘异常I/O

Postgres query: Abnormal Disk I/O

postgresql

database-performance