ElasticSearch 备份到 S3 AWS

Question

我正在尝试将 AWS 上的 ElasticSearch 集群备份到 S3 存储桶。

我关注了以下 'tutorial' ：Use Amazon S3 to Store a Single Amazon Elasticsearch Service Index

这些是我采取的步骤：

创建 S3 存储桶（称为 cb-search-es-backup）。

创建新策略（名为 P_ES_SNAPSHOT_TO_S3）：

{
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation",
                "s3:ListBucketMultipartUploads",
                "s3:ListBucketVersions"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::cb-search-es-backup"
            ]
        },
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::cb-search-es-backup/*"
            ]
        }
    ],
    "Version": "2012-10-17"
}

创建一个服务角色，将之前创建的策略附加到它 arn:aws:iam::12345678910:role/Role_ES_TO_S3

角色的信任策略：

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    },
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "es.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

在 Kibana 中，我尝试在开发工具中使用以下内容：

PUT /_snapshot/ES_BACKUP
{
"type": "s3",
    "settings": {
        "bucket": "cb-search-es-backup",
        "region": "eu-west-1",
        "role_arn": "arn:aws:iam::423628447134:role/Role_ES_TO_S3"
    }
}

但我收到了来自 kibana 的以下回复：

{ "Message": "User: anonymous is not authorized to perform: iam:PassRole on resource: arn:aws:iam::12345678910:role/Role_ES_TO_S3" }

Answer 1

我在使用 kibana 时遇到了同样的错误。尝试使用 aws-es-proxy.

进行备份

如果您要使用 docker 图片，请按以下方式传递 AWS 访问凭据：

docker run --rm -it -p 9200:9200 -e "AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXX" -e "AWS_SECRET_ACCESS_KEY=YYYYYYYYYYYYYYYYYYYYYYYYY" abutaha/aws-es-proxy ./aws-es-proxy -verbose -listen 0.0.0.0:9200 -endpoint https://enpoint-url

ElasticSearch 备份到 S3 AWS

ElasticSearch backup to S3 AWS

amazon-s3

amazon-web-services

elasticsearch

kibana