SQL 服务器中缺少谓词的筛选索引无法按预期工作

Question

我目前正在 SQL 服务器中试验过滤索引。我试图通过将 BOL 中的以下提示付诸实践来缩小过滤后的索引：

A column in the filtered index expression does not need to be a key or included column in the filtered index definition if the filtered index expression is equivalent to the query predicate and the query does not return the column in the filtered index expression with the query results.

我已经在一个小的测试脚本中重现了这个问题：我的 table 看起来如下：

CREATE TABLE #test
(
    ID  BIGINT NOT NULL IDENTITY(1,1),
    ARCHIVEDATE DATETIME NULL,
    CLOSINGDATE DATETIME NULL,
    OBJECTTYPE INTEGER NOT NULL,
    ACTIVE BIT NOT NULL,
    FILLER1 CHAR(255) DEFAULT 'just a filler',
    FILLER2 CHAR(255) DEFAULT 'just a filler',
    FILLER3 CHAR(255) DEFAULT 'just a filler',
    FILLER4 CHAR(255) DEFAULT 'just a filler',
    FILLER5 CHAR(255) DEFAULT 'just a filler',
    CONSTRAINT test_pk PRIMARY KEY CLUSTERED (ID ASC)
);

我需要优化以下查询：

SELECT  
    COUNT(*) 
FROM    
    #test 
WHERE       
        ARCHIVEDATE IS NULL 
    AND CLOSINGDATE IS NOT NULL 
    AND ISNULL(ACTIVE,1) != 0

因此我建立了以下过滤索引：

CREATE NONCLUSTERED INDEX idx_filterTest ON #test (/*ARCHIVEDATE ASC,*/CLOSINGDATE ASC) INCLUDE (ACTIVE) WHERE ARCHIVEDATE IS NULL;

ARCHIVEDATE 已经在过滤器中，不会在 SELECT 中使用，因此它不包含在索引键或包含中。

但是，如果我运行查询我得到以下计划：

ARCHIVEDATE 的聚集索引中有一个键查找。为什么？我已在 SQL Server 2008 和 SQL Server 2016 上重现此行为。

如果我在键中使用 ARCHIVEDATE 创建索引，我只需进行索引查找即可。所以在我看来，BOL 中的这一段并不总是适用。

这是我完整的重现脚本：

--DROP TABLE #test;
CREATE TABLE #test
(
    ID  BIGINT NOT NULL IDENTITY(1,1),
    ARCHIVEDATE DATETIME NULL,
    CLOSINGDATE DATETIME NULL,
    OBJECTTYPE INTEGER NOT NULL,
    ACTIVE BIT NOT NULL,
    FILLER1 CHAR(255) DEFAULT 'just a filler',
    FILLER2 CHAR(255) DEFAULT 'just a filler',
    FILLER3 CHAR(255) DEFAULT 'just a filler',
    FILLER4 CHAR(255) DEFAULT 'just a filler',
    FILLER5 CHAR(255) DEFAULT 'just a filler',
    CONSTRAINT test_pk PRIMARY KEY CLUSTERED (ID ASC)
);



INSERT INTO #test
(ARCHIVEDATE, CLOSINGDATE, OBJECTTYPE, ACTIVE)
SELECT TOP 200
    NULL,
    dates.calcDate,
    4711,
    dates.number%2
FROM
    (
        SELECT
            /* Erzeugen des Datums durch Addieren der jeweiligen Sequenznummer zum StartDate */
            DATEADD(DAY, seq.number, '20120101') AS calcDate, number
        FROM
        (
            /* Abfrage zur Erstellung einer Nummernsequenz von 0 bis 9999. Dient als Basis zur Aufbereitung aller Datumswerte im Zeitraum. Die Sequenz reicht für einen Zeitraum von ca. 30 Jahren aus. */
            SELECT
                a.num * 1000 + b.num * 100 + c.num * 10 + d.num AS number
            FROM
                        ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) c
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) d
        ) seq 
        WHERE
            /* Einschränkung der Nummernsequenz auf die Anzahl der Tage im gewünschten Aufbereitungszeitraum */
            seq.number <= 5000
    ) dates
ORDER BY
    dates.number
;



INSERT INTO #test
(ARCHIVEDATE, CLOSINGDATE, OBJECTTYPE, ACTIVE)
SELECT TOP 1000
    dates.calcDate + 3,
    dates.calcDate,
    4711,
    dates.number%2
FROM
    (
        SELECT
            /* Erzeugen des Datums durch Addieren der jeweiligen Sequenznummer zum StartDate */
            DATEADD(DAY, seq.number, '20120101') AS calcDate, number
        FROM
        (
            /* Abfrage zur Erstellung einer Nummernsequenz von 0 bis 9999. Dient als Basis zur Aufbereitung aller Datumswerte im Zeitraum. Die Sequenz reicht für einen Zeitraum von ca. 30 Jahren aus. */
            SELECT
                a.num * 1000 + b.num * 100 + c.num * 10 + d.num AS number
            FROM
                        ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) c
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) d
        ) seq 
        WHERE
            /* Einschränkung der Nummernsequenz auf die Anzahl der Tage im gewünschten Aufbereitungszeitraum */
            seq.number <= 5000
    ) dates
ORDER BY
    dates.number
;


INSERT INTO #test
(ARCHIVEDATE, CLOSINGDATE, OBJECTTYPE, ACTIVE)
SELECT TOP 100000
    dates.calcDate,
    NULL,
    4711,
    dates.number%2
FROM
    (
        SELECT
            /* Erzeugen des Datums durch Addieren der jeweiligen Sequenznummer zum StartDate */
            DATEADD(DAY, seq.number, '20120101') AS calcDate, number
        FROM
        (
            /* Abfrage zur Erstellung einer Nummernsequenz von 0 bis 9999. Dient als Basis zur Aufbereitung aller Datumswerte im Zeitraum. Die Sequenz reicht für einen Zeitraum von ca. 30 Jahren aus. */
            SELECT
                a.num * 1000 + b.num * 100 + c.num * 10 + d.num AS number
            FROM
                        ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) a
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) b
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) c
            CROSS JOIN  ( SELECT 0 AS num UNION ALL SELECT 1 AS num UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9) d
        ) seq 
        WHERE
            /* Einschränkung der Nummernsequenz auf die Anzahl der Tage im gewünschten Aufbereitungszeitraum */
            seq.number <= 5000
    ) dates
ORDER BY
    dates.number
;


--DROP INDEX idx_filterTest ON #test;
--CREATE NONCLUSTERED INDEX idx_filterTest ON #test (ARCHIVEDATE ASC,CLOSINGDATE ASC) INCLUDE (ACTIVE) WHERE ARCHIVEDATE IS NULL;
CREATE NONCLUSTERED INDEX idx_filterTest ON #test (/*ARCHIVEDATE ASC,*/CLOSINGDATE ASC) INCLUDE (ACTIVE) WHERE ARCHIVEDATE IS NULL;



SELECT  
    COUNT(*) 
FROM    
    #test 
WHERE       
        ARCHIVEDATE IS NULL 
    AND CLOSINGDATE IS NOT NULL 
    AND ISNULL(ACTIVE,1) != 0;

Answer 1

这是优化器中的一个错误，特别是它处理 IS NULL 过滤器的方式。这是一个更简单的重现：

CREATE TABLE #T(ID INT IDENTITY PRIMARY KEY, X INT);
INSERT #T(X) SELECT TOP(10000) message_id FROM sys.messages WHERE message_id <> 1;
INSERT #T(X) VALUES (1);
INSERT #T(X) VALUES (NULL);
CREATE INDEX IX_#T_X_null ON #T(ID) WHERE X IS NULL;
CREATE INDEX IX_#T_X_1 ON #T(ID) WHERE X = 1;

显然 IX_#T_X_null 涵盖了以下查询：

SELECT MIN(ID) FROM #T WHERE X IS NULL;

并且优化器确实选择了它，但是我们得到了一个执行计划，其中插入了多余的聚簇索引查找。但是：

SELECT MIN(ID) FROM #T WHERE X = 1;

现在我们得到一个没有聚集索引查找的查询。当涉及 IS NULL 时，优化器似乎认识到过滤索引适用，但无法将条件传播到后面的步骤。如果我们包含带有索引的列，我们可以清楚地看到这一点：

CREATE INDEX IX_#T_X_null ON #T(ID, X) WHERE X IS NULL;

如果您现在比较 WHERE X = 1 和 WHERE X IS NULL 查询的执行计划，您会发现在 X IS NULL 的情况下，优化器将谓词添加到索引中扫描，它与 X = 1.

无关

再深入一点，通过这个特定的设置，您会发现这是一个 known issue, already reported on Connect。然而，根据 Microsoft 的说法，"this is actually not a bug but rather a known gap in functionality"（我认为这在技术上是正确的，因为结果并没有错，它只是没有达到应有的性能）。另外，"this is now an active DCR for a future release of SQL Server"，但那是 6 年前的事了，票据已关闭 "Won't Fix" -- 所以不要屏住呼吸。

不幸的是，解决方法确实是将该列包含在索引中——我将其设为包含列而不是键，因为这会增加非叶级别的开销：

CREATE NONCLUSTERED INDEX idx_filterTest ON #test (CLOSINGDATE ASC)
INCLUDE (ACTIVE, ARCHIVEDATE) 
WHERE ARCHIVEDATE IS NULL;

我说 "unfortunately" 因为这个总是 NULL 的列仍然会毫无意义地占用行 space（因为 DATETIME 是固定大小的数据类型）。即便如此，它可能比从聚簇索引查找中获得额外的 I/O 要好得多。此外，compressing the index 可以将开销减少到几乎为零（甚至行压缩也可以）。

SQL 服务器中缺少谓词的筛选索引无法按预期工作

Filtered Index in SQL Server missing predicate does not work as expected

sql-server

indexing

filtered-index