动态映射数值字段数据类型大于 2^63 - 1（long 类型）

Question

我有以下示例输入数据 "doc"。

使用 python elasticsearch 建立索引：

doc =  {
    "node": [{
        "table": [{
          "node-table": {
            "packets-up": 18440044073709951615,
            "packets-down": 18447644073709991615
          }
        }]
      }]
}

from elasticsearch import Elasticsearch

es = Elasticsearch(hosts="localhost:9200")

res = es.indices.create(index="doc")

es.index(index="doc", doc_type='docs', body=doc)

尝试使用动态映射索引数据时，出现以下错误：

    Traceback (most recent call last):
  File "test.py", line 60, in <module>
    es.index(index="doc_test", doc_type='docs', body=doc)
  File "/Users/user/Projects/2018/es_test/.env/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/Users/user/Projects/2018/es_test/.env/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 319, in index
    _make_path(index, doc_type, id), params=params, body=body)
  File "/Users/user/Projects/2018/es_test/.env/lib/python2.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/Users/user/Projects/2018/es_test/.env/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 186, in perform_request
    self._raise_error(response.status, raw_data)
  File "/Users/user/Projects/2018/es_test/.env/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: RequestError(400, u'mapper_parsing_exception', u'failed to parse')

我假设这是由于 "long" 数据类型不支持的数值。

我们如何处理这些数值。

ElasticSearch 跟踪：

curl -H 'Content-Type: application/json' -XPOST 'http://localhost:9200/doc/docs?pretty' -d '
  {
    "node": [
      {
        "table": [
          {
            "node-table": {
              "packets-down": 18447644073709991615,
              "packets-up": 18440044073709951615
            }
          }
        ]
      }
    ]
  }
'

回复：

    {
  "error" : {
    "root_cause" : [
      {
        "type" : "mapper_parsing_exception",
        "reason" : "failed to parse"
      }
    ],
    "type" : "mapper_parsing_exception",
    "reason" : "failed to parse",
    "caused_by" : {
      "type" : "illegal_state_exception",
      "reason" : "No matching token for number_type [BIG_INTEGER]"
    }
  },
  "status" : 400
}

Answer 1

你有两种选择来避免这个问题。

选项 A。将这些值存储为 float（或 double）而不是 long。

首先您需要确保您的 packets-down 和 packets-up 字段映射为 float（或 double），如下所示：

PUT doc_test
{
  "mappings": {
    "docs": {
      "dynamic_templates": [
        {
          "bignums": {
            "match": "packets*",
            "mapping": {
              "type": "float"
            }
          }
        }
      ]
    }
  }
}

然后您需要将数字用双引号括起来并将它们作为字符串发送：

doc =  {
    "node": [{
        "table": [{
          "node-table": {
            "packets-up": "18440044073709951615",
            "packets-down": "18447644073709991615"
          }
        }]
      }]
}

这会起作用，您将能够将您的数据包字段与包含数值的任何其他字段相加。

选项 B. 启用 numeric detection（默认禁用）

PUT doc_test
{
  "mappings": {
    "docs": {
      "numeric_detection": true
    }
  }
}

然后您还需要将数字用双引号括起来并作为字符串发送：

doc =  {
    "node": [{
        "table": [{
          "node-table": {
            "packets-up": "18440044073709951615",
            "packets-down": "18447644073709991615"
          }
        }]
      }]
}

因此，大数字将被映射为 float

动态映射数值字段数据类型大于 2^63 - 1（long 类型）

Dynamic mapping numeric field datatypes greater than 2^63 - 1 (long type)

mapping

elasticsearch

long-integer