运行 嵌套的 cloudformation 堆栈作为另一个角色
Run nested cloudformation stack as another role
我正在尝试构建一个将部署 EKS 集群、节点组和工作负载的 cloudformation 模板。
使用以下 lambda layer，我创建了一个可以与 EKS 集群交互的函数；但是，这仅在函数承担创建集群的用户的角色时有效。
我发现的一个问题是无法在 SSO 环境中承担 SSO 用户的角色，因为 AWS 管理信任策略。
如果我在创建集群之前担任另一个角色并让 lambda 担任该角色，该函数将起作用。
遗憾的是，无法传入用于创建集群的特定角色，RoleArn 仅提供控制平面与其他 AWS 服务交互的权限。
我想知道是否可以创建一个嵌套的堆栈结构，这样做会怎样？
- 在主栈中创建一个角色
- 然后调用嵌套模板承担新角色
- 在子堆栈中将创建一个 EKS 集群
- 在主堆栈中将创建并调用 lambda 函数
这在技术上可行吗？
作为参考，这是函数当前正在执行的操作。
def update_kubeconfig(clusterName, role):
runCmd("aws eks update-kubeconfig --name {} --kubeconfig /tmp/kubeconfig --role-arn {}".format(clusterName, role))
def getPods():
runCmd("kubectl get pod --kubeconfig /tmp/kubeconfig")
update_kubeconfig('eks-cluster-1', 'arn:aws:iam::3088564456:role/cluster-admin')
我能够通过在主堆栈中创建和调用一个 lambda 函数来解决这个问题，该函数在担任 eks 集群管理员角色后创建了一个子堆栈。
为了避免在子堆栈中创建 IAM 角色，我在主堆栈中创建了所有这些角色，然后将 ARN 传递到子堆栈中。
我希望这对需要做类似事情的其他人有用
主堆栈
EKSClusterRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- eks.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/AmazonEKSClusterPolicy'
DeployCloudformationStackLambdaRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: AllowRolePassAndCF
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
- 'iam:PassRole'
- 'iam:GetRole'
- 'cloudformation:CreateStack'
- 'cloudformation:CreateChangeSet'
- 'eks:DescribeCluster'
Resource: '*'
EKSClusterAdminRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
AWS: !GetAtt DeployCloudformationStackLambdaRole.Arn
Action: sts:AssumeRole
Policies:
- PolicyName: RunKubeCtlCommands
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'eks:*'
- 'cloudformation:CreateStack'
- 'cloudformation:CreateChangeSet'
- 'serverlessrepo:CreateCloudFormationTemplate'
- 'serverlessrepo:GetCloudFormationTemplate'
- 's3:GetObject'
- 'lambda:PublishLayerVersion'
- 'lambda:CreateFunction'
- 'lambda:GetLayerVersion'
- 'lambda:GetFunction'
- 'lambda:InvokeFunction'
- 'lambda:GetFunctionConfiguration'
Resource: '*'
- Effect: Allow
Action:
- 'iam:PassRole'
- 'iam:GetRole'
Resource:
- !GetAtt EKSClusterRole.Arn
- !GetAtt DeployCloudformationStackLambdaRole.Arn
TriggerStack:
Type: "Custom::TriggerStack"
DependsOn: DeployCloudformationStack
Properties:
ServiceToken: !GetAtt DeployCloudformationStack.Arn
cfStackName: "eksDeploy"
assumeRoleARN: !GetAtt EKSClusterAdminRole.Arn
templateUrl: !Sub 'https://test-${AWS::Region}-public-lambda.s3.amazonaws.com/DeployEKS.yml'
stackParameters:
- ParameterKey: EksClusterName
ParameterValue: "test-cluster"
- ParameterKey: EksClusterRole
ParameterValue: !GetAtt EKSClusterRole.Arn
- ParameterKey: EksSubnets
ParameterValue: "subnet-256faaf,subnet-6e205960"
- ParameterKey: EksClusterAdminRole
ParameterValue: !GetAtt EKSClusterAdminRole.Arn
- ParameterKey: KubeLambdaRole
ParameterValue: !GetAtt DeployCloudformationStackLambdaRole.Arn
DeployCloudformationStack:
Type: AWS::Lambda::Function
Properties:
Description: Deploy cloudformation stack
Handler: index.lambda_handler
Runtime: python3.8
Role: !GetAtt DeployCloudformationStackLambdaRole.Arn
MemorySize: 128
Timeout: 30
Code:
ZipFile: |
import cfnresponse
import json, os, boto3, logging
from botocore.exceptions import ClientError
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
payload = ""
result = cfnresponse.SUCCESS
logger = logging.getLogger()
logger.setLevel(logging.INFO)
try:
if event['RequestType'] == 'Create':
payload = deployStack(event['ResourceProperties'])
except ClientError as e:
logger.error('Error: %s', e)
result = cfnresponse.FAILED
cfnresponse.send(event, context, result, payload)
def deployStack(input):
sts = boto3.client('sts').assume_role(RoleArn=input['assumeRoleARN'],RoleSessionName="lambda_assume_role")
client = boto3.client('cloudformation',
aws_access_key_id=sts['Credentials']['AccessKeyId'],
aws_secret_access_key=sts['Credentials']['SecretAccessKey'],
aws_session_token=sts['Credentials']['SessionToken']
)
response = client.create_stack(StackName=input['cfStackName'], TemplateURL=input['templateUrl'],Parameters=input['stackParameters'], TimeoutInMinutes=60, Capabilities=['CAPABILITY_IAM','CAPABILITY_NAMED_IAM','CAPABILITY_AUTO_EXPAND'])
子堆栈
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Parameters:
KubeLambdaRole:
Default: arn:*
Type: String
EksClusterRole:
Default: arn:*
Type: String
EksSubnets:
Description: Subnet IDs
Type: CommaDelimitedList
EksClusterName:
Default: pebble-oceans
Type: String
EksClusterAdminRole:
Default: arn:*
Type: String
Resources:
KubeCtlLayer:
Type: AWS::Serverless::Application
Properties:
Parameters:
LayerName: kubelayer
Location:
ApplicationId: arn:aws:serverlessrepo:us-east-1:903779448426:applications/lambda-layer-kubectl
SemanticVersion: 2.0.0
EKS:
Type: 'AWS::EKS::Cluster'
Properties:
Name: !Ref EksClusterName
Version: '1.19'
RoleArn: !Ref EksClusterRole
ResourcesVpcConfig:
SubnetIds: !Ref EksSubnets
TriggerKubectl:
Type: "Custom::TriggerKubectl"
DependsOn:
- KubeLambda
- EKS
Properties:
ServiceToken: !GetAtt KubeLambda.Arn
clusterName: !Ref EksClusterName
roleArn: !Ref EksClusterAdminRole
region: !Ref "AWS::Region"
KubeLambda:
Type: AWS::Lambda::Function
Properties:
Description: Copies images into ECR
Handler: index.lambda_handler
Runtime: python2.7
Layers:
- !GetAtt KubeCtlLayer.Outputs.LayerVersionArn
Role: !Ref KubeLambdaRole
MemorySize: 512
Timeout: 300
Code:
ZipFile: |
import cfnresponse
import json, os, boto3, logging
from botocore.exceptions import ClientError
from subprocess import Popen, PIPE, STDOUT
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
payload = ""
result = cfnresponse.SUCCESS
logger = logging.getLogger()
logger.setLevel(logging.INFO)
try:
if event['RequestType'] == 'Create':
clusterName=event['ResourceProperties']['clusterName']
roleArn=cluster=event['ResourceProperties']['roleArn']
region=cluster=event['ResourceProperties']['region']
update_kubeconfig(clusterName, region, roleArn)
payload = getPods()
except ClientError as e:
logger.error('Error: %s', e)
result = cfnresponse.FAILED
cfnresponse.send(event, context, result, payload)
def update_kubeconfig(clusterName, region, role):
runCmd("aws eks update-kubeconfig --name {} --region {} --role-arn {} --kubeconfig /tmp/kubeconfig".format(clusterName, region, role))
def getPods():
runCmd("kubectl get pod --kubeconfig /tmp/kubeconfig")
def runCmd(cmd):
my_env = os.environ.copy()
my_env["PATH"] = my_env["PATH"] + ":/opt/awscli:/opt/kubectl"
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True, env=my_env)
output = p.stdout.read()
print output
我正在尝试构建一个将部署 EKS 集群、节点组和工作负载的 cloudformation 模板。
使用以下 lambda layer，我创建了一个可以与 EKS 集群交互的函数；但是，这仅在函数承担创建集群的用户的角色时有效。
我发现的一个问题是无法在 SSO 环境中承担 SSO 用户的角色，因为 AWS 管理信任策略。 如果我在创建集群之前担任另一个角色并让 lambda 担任该角色，该函数将起作用。
遗憾的是，无法传入用于创建集群的特定角色，RoleArn 仅提供控制平面与其他 AWS 服务交互的权限。
我想知道是否可以创建一个嵌套的堆栈结构，这样做会怎样？
- 在主栈中创建一个角色
- 然后调用嵌套模板承担新角色
- 在子堆栈中将创建一个 EKS 集群
- 在主堆栈中将创建并调用 lambda 函数
这在技术上可行吗？
作为参考，这是函数当前正在执行的操作。
def update_kubeconfig(clusterName, role):
runCmd("aws eks update-kubeconfig --name {} --kubeconfig /tmp/kubeconfig --role-arn {}".format(clusterName, role))
def getPods():
runCmd("kubectl get pod --kubeconfig /tmp/kubeconfig")
update_kubeconfig('eks-cluster-1', 'arn:aws:iam::3088564456:role/cluster-admin')
我能够通过在主堆栈中创建和调用一个 lambda 函数来解决这个问题，该函数在担任 eks 集群管理员角色后创建了一个子堆栈。
为了避免在子堆栈中创建 IAM 角色，我在主堆栈中创建了所有这些角色，然后将 ARN 传递到子堆栈中。
我希望这对需要做类似事情的其他人有用
主堆栈
EKSClusterRole:
Type: 'AWS::IAM::Role'
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- eks.amazonaws.com
Action:
- 'sts:AssumeRole'
ManagedPolicyArns:
- 'arn:aws:iam::aws:policy/AmazonEKSClusterPolicy'
DeployCloudformationStackLambdaRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: sts:AssumeRole
Policies:
- PolicyName: AllowRolePassAndCF
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'logs:CreateLogGroup'
- 'logs:CreateLogStream'
- 'logs:PutLogEvents'
- 'iam:PassRole'
- 'iam:GetRole'
- 'cloudformation:CreateStack'
- 'cloudformation:CreateChangeSet'
- 'eks:DescribeCluster'
Resource: '*'
EKSClusterAdminRole:
Type: AWS::IAM::Role
Properties:
Path: /
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
AWS: !GetAtt DeployCloudformationStackLambdaRole.Arn
Action: sts:AssumeRole
Policies:
- PolicyName: RunKubeCtlCommands
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- 'eks:*'
- 'cloudformation:CreateStack'
- 'cloudformation:CreateChangeSet'
- 'serverlessrepo:CreateCloudFormationTemplate'
- 'serverlessrepo:GetCloudFormationTemplate'
- 's3:GetObject'
- 'lambda:PublishLayerVersion'
- 'lambda:CreateFunction'
- 'lambda:GetLayerVersion'
- 'lambda:GetFunction'
- 'lambda:InvokeFunction'
- 'lambda:GetFunctionConfiguration'
Resource: '*'
- Effect: Allow
Action:
- 'iam:PassRole'
- 'iam:GetRole'
Resource:
- !GetAtt EKSClusterRole.Arn
- !GetAtt DeployCloudformationStackLambdaRole.Arn
TriggerStack:
Type: "Custom::TriggerStack"
DependsOn: DeployCloudformationStack
Properties:
ServiceToken: !GetAtt DeployCloudformationStack.Arn
cfStackName: "eksDeploy"
assumeRoleARN: !GetAtt EKSClusterAdminRole.Arn
templateUrl: !Sub 'https://test-${AWS::Region}-public-lambda.s3.amazonaws.com/DeployEKS.yml'
stackParameters:
- ParameterKey: EksClusterName
ParameterValue: "test-cluster"
- ParameterKey: EksClusterRole
ParameterValue: !GetAtt EKSClusterRole.Arn
- ParameterKey: EksSubnets
ParameterValue: "subnet-256faaf,subnet-6e205960"
- ParameterKey: EksClusterAdminRole
ParameterValue: !GetAtt EKSClusterAdminRole.Arn
- ParameterKey: KubeLambdaRole
ParameterValue: !GetAtt DeployCloudformationStackLambdaRole.Arn
DeployCloudformationStack:
Type: AWS::Lambda::Function
Properties:
Description: Deploy cloudformation stack
Handler: index.lambda_handler
Runtime: python3.8
Role: !GetAtt DeployCloudformationStackLambdaRole.Arn
MemorySize: 128
Timeout: 30
Code:
ZipFile: |
import cfnresponse
import json, os, boto3, logging
from botocore.exceptions import ClientError
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
payload = ""
result = cfnresponse.SUCCESS
logger = logging.getLogger()
logger.setLevel(logging.INFO)
try:
if event['RequestType'] == 'Create':
payload = deployStack(event['ResourceProperties'])
except ClientError as e:
logger.error('Error: %s', e)
result = cfnresponse.FAILED
cfnresponse.send(event, context, result, payload)
def deployStack(input):
sts = boto3.client('sts').assume_role(RoleArn=input['assumeRoleARN'],RoleSessionName="lambda_assume_role")
client = boto3.client('cloudformation',
aws_access_key_id=sts['Credentials']['AccessKeyId'],
aws_secret_access_key=sts['Credentials']['SecretAccessKey'],
aws_session_token=sts['Credentials']['SessionToken']
)
response = client.create_stack(StackName=input['cfStackName'], TemplateURL=input['templateUrl'],Parameters=input['stackParameters'], TimeoutInMinutes=60, Capabilities=['CAPABILITY_IAM','CAPABILITY_NAMED_IAM','CAPABILITY_AUTO_EXPAND'])
子堆栈
AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Parameters:
KubeLambdaRole:
Default: arn:*
Type: String
EksClusterRole:
Default: arn:*
Type: String
EksSubnets:
Description: Subnet IDs
Type: CommaDelimitedList
EksClusterName:
Default: pebble-oceans
Type: String
EksClusterAdminRole:
Default: arn:*
Type: String
Resources:
KubeCtlLayer:
Type: AWS::Serverless::Application
Properties:
Parameters:
LayerName: kubelayer
Location:
ApplicationId: arn:aws:serverlessrepo:us-east-1:903779448426:applications/lambda-layer-kubectl
SemanticVersion: 2.0.0
EKS:
Type: 'AWS::EKS::Cluster'
Properties:
Name: !Ref EksClusterName
Version: '1.19'
RoleArn: !Ref EksClusterRole
ResourcesVpcConfig:
SubnetIds: !Ref EksSubnets
TriggerKubectl:
Type: "Custom::TriggerKubectl"
DependsOn:
- KubeLambda
- EKS
Properties:
ServiceToken: !GetAtt KubeLambda.Arn
clusterName: !Ref EksClusterName
roleArn: !Ref EksClusterAdminRole
region: !Ref "AWS::Region"
KubeLambda:
Type: AWS::Lambda::Function
Properties:
Description: Copies images into ECR
Handler: index.lambda_handler
Runtime: python2.7
Layers:
- !GetAtt KubeCtlLayer.Outputs.LayerVersionArn
Role: !Ref KubeLambdaRole
MemorySize: 512
Timeout: 300
Code:
ZipFile: |
import cfnresponse
import json, os, boto3, logging
from botocore.exceptions import ClientError
from subprocess import Popen, PIPE, STDOUT
def lambda_handler(event, context):
print("Received event: " + json.dumps(event, indent=2))
payload = ""
result = cfnresponse.SUCCESS
logger = logging.getLogger()
logger.setLevel(logging.INFO)
try:
if event['RequestType'] == 'Create':
clusterName=event['ResourceProperties']['clusterName']
roleArn=cluster=event['ResourceProperties']['roleArn']
region=cluster=event['ResourceProperties']['region']
update_kubeconfig(clusterName, region, roleArn)
payload = getPods()
except ClientError as e:
logger.error('Error: %s', e)
result = cfnresponse.FAILED
cfnresponse.send(event, context, result, payload)
def update_kubeconfig(clusterName, region, role):
runCmd("aws eks update-kubeconfig --name {} --region {} --role-arn {} --kubeconfig /tmp/kubeconfig".format(clusterName, region, role))
def getPods():
runCmd("kubectl get pod --kubeconfig /tmp/kubeconfig")
def runCmd(cmd):
my_env = os.environ.copy()
my_env["PATH"] = my_env["PATH"] + ":/opt/awscli:/opt/kubectl"
p = Popen(cmd, shell=True, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True, env=my_env)
output = p.stdout.read()
print output