使用 Cloudformation 模板创建特定资源的多个实例
Create multiple instances of a specific resource using Cloud Formation Template
提前致谢!
我正在尝试使用云形成模板创建 3 个单独的 Ec2 实例,实例 1 运行 spark master,实例 2 有 spark worker,实例 3 有客户端 运行。
我希望 spark master 有一个实例,但 spark worker 有多个实例,spark master 主机作为用户数据参数,我该怎么做?
此致,
阿梅
除非您有非常具体的理由不这样做,否则我不建议使用 EC2 和 CloudFormation 从头开始创建 Apache Spark 集群。相反,我会考虑使用 Amazon EMR, which supports Spark out of the box, and will save you from repeating a large amount of custom setup getting a Spark cluster configured and provisioned correctly. Using EMR, you can easily choose the number and type of instances, or even auto scale depending on your cluster's current load. Though using the Console is best for getting started, CloudFormation also supports EMR through its AWS::EMR::Cluster
资源设置一个 Spark 集群。
如果您坚持使用 CloudFormation 直接启动 EC2 实例,请查看工作池的 AWS::EC2::Instance
resource for your master/client, and AWS::AutoScaling::AutoScalingGroup
资源。
根据 AWS CloudFormation 文档:
You must declare each resource separately; however, if you have
multiple resources of the same type, you can declare them together by
separating them with commas.
要传递用户数据,您可以使用资源的 UserData
属性。
参考: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html
编辑 2019-07-15: 看来他们已经更新了 AWS 文档并删除了我引用的那个片段。您必须在 CloudFormation 中单独定义每个 EC2 实例。
提前致谢! 我正在尝试使用云形成模板创建 3 个单独的 Ec2 实例,实例 1 运行 spark master,实例 2 有 spark worker,实例 3 有客户端 运行。
我希望 spark master 有一个实例,但 spark worker 有多个实例,spark master 主机作为用户数据参数,我该怎么做?
此致, 阿梅
除非您有非常具体的理由不这样做,否则我不建议使用 EC2 和 CloudFormation 从头开始创建 Apache Spark 集群。相反,我会考虑使用 Amazon EMR, which supports Spark out of the box, and will save you from repeating a large amount of custom setup getting a Spark cluster configured and provisioned correctly. Using EMR, you can easily choose the number and type of instances, or even auto scale depending on your cluster's current load. Though using the Console is best for getting started, CloudFormation also supports EMR through its AWS::EMR::Cluster
资源设置一个 Spark 集群。
如果您坚持使用 CloudFormation 直接启动 EC2 实例,请查看工作池的 AWS::EC2::Instance
resource for your master/client, and AWS::AutoScaling::AutoScalingGroup
资源。
根据 AWS CloudFormation 文档:
You must declare each resource separately; however, if you have multiple resources of the same type, you can declare them together by separating them with commas.
要传递用户数据,您可以使用资源的 UserData
属性。
参考: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/resources-section-structure.html
编辑 2019-07-15: 看来他们已经更新了 AWS 文档并删除了我引用的那个片段。您必须在 CloudFormation 中单独定义每个 EC2 实例。