如何应用自定义分析器？

Question

刚刚发现我们的 Elastic Search 存在问题。它不会为字段名称中的“&”返回任何内容。做了一些谷歌搜索，我想我需要一个自定义分析器。以前从未使用过 ES，假设我在这里缺少一些基本的东西。

这就是我得到的，但没有按预期工作。

PUT custom_analyser
{
 "settings": {
    "analysis": {
      "analyzer": {
        "suggest_analyzer": {
          "type":      "custom",
          "tokenizer": "whitespace",
          "filter":    [ "lowercase", "my_synonym_filter" ]
        }
      },
      "filter": {
        "my_synonym_filter": {
          "type": "synonym", 
          "synonyms": [
              "&, and",
              "foo, bar" ]
        }
      }
    }
  }
}

并尝试像这样使用它：

GET custom_analyser/_search
{
  "aggs": {
    "section": {
      "terms": {
        "field": "section",
        "size": 10,
        "shard_size": 500,
        "include": "jill & jerry" //Not returning anything back for this field using default analyser
      }
    }
  }
}

输出：

{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": 0,
    "max_score": null,
    "hits": []
  },
  "aggregations": {
    "section": {
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": []
    }
  }
}

映射

 "_doc":{  
      "dynamic":"false",
      "date_detection":false,
      "properties":{  
         "section":{  
            "type":"keyword"
         }
      }
   }

获取 custom_analyser:

{
  "custom_analyser": {
    "aliases": {},
    "mappings": {},
    "settings": {
      "index": {
        "number_of_shards": "5",
        "provided_name": "custom_analyser",
        "creation_date": "1565971369814",
        "analysis": {
          "filter": {
            "my_synonym_filter": {
              "type": "synonym",
              "synonyms": [
                "&, and",
                "foo, bar"
              ]
            }
          },
          "analyzer": {
            "suggest_analyzer": {
              "filter": [
                "lowercase",
                "my_synonym_filter"
              ],
              "type": "custom",
              "tokenizer": "whitespace"
            }
          }
        },
        "number_of_replicas": "1",
        "uuid": "oVMOU5wPQ--vKhE3dDFG2Q",
        "version": {
          "created": "6030199"
        }
      }
    }
  }
}

Answer 1

我认为这里有点混乱：分析器不会帮助您，因为您（正确地）使用 keyword 字段进行聚合，但没有分析这些字段。您只能在这些字段上使用 normalizer。

对于您的特定问题：include (and exclude) are regular expressions — 您需要转义 & 才能按预期进行。

完整示例

映射和示例数据：

PUT test
{
  "mappings": {
    "properties": {
      "section": {
        "type": "keyword"
      }
    }
  }
}

PUT test/_doc/1
{
  "section": "jill & jerry"
}
PUT test/_doc/2
{
  "section": "jill jerry"
}
PUT test/_doc/3
{
  "section": "jill"
}
PUT test/_doc/4
{
  "section": "jill & jerry"
}

查询 — 您需要双反斜杠才能在此处进行转义（并且我还排除了带有 "size": 0 的实际文档以保持响应更短）：

GET test/_search
{
  "size": 0,
  "aggs": {
    "section": {
      "terms": {
        "field": "section",
        "include": "jill \& jerry"
      }
    }
  }
}

回复：

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 4,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  },
  "aggregations" : {
    "section" : {
      "doc_count_error_upper_bound" : 0,
      "sum_other_doc_count" : 0,
      "buckets" : [
        {
          "key" : "jill & jerry",
          "doc_count" : 2
        }
      ]
    }
  }
}

如何应用自定义分析器？

How to apply custom analyser?

elasticsearch

kibana

完整示例