小西秀和です。
本題に入る前にSNSで反応があったので、本当は記事を一通り書いてからまとめで書く予定でしたが、多段階の承認フローのシステム化に関して書いている意図をこちらで説明しておきます。
承認フローについて記事を書き始めた背景にはChatGPT(GPT-4)の登場があります。
個人的には人間がおこなう不要な多段階の承認フローはなくすべきだと考えています。
ただ、一方で承認フローは承認に必要な知識、分析能力、権限をもつ者が意思決定をおこなえる最後の砦であるとも言えます。
そのため、次のような考えから現在、人間がアナログベースでおこなっている承認フローはAPIが介入できる形でシステム化するべきではないかという実験的な意図で多段階の承認フローに関する記事を書いています。
- 承認フローをAPIが介入できる形でシステム化すれば人間でもChatGPTでも意思決定のステップを柔軟に切り替えられる
- 最初は人間が承認し、ChatGPTの承認能力が実用に足る場合はChatGPTに任せることも可能
- ChatGPTに不安要素がある、または最後の砦として人間が意思決定をしたい場合は人間が内容を確認して承認する
- 承認精度を上げるためにChatGPTと人間で多段階の承認フローを組む
ちなみに余談ですが、AWS CDKを使用せずにAWS CloudFormationテンプレートを使用している理由は、この規模のデモであればテンプレートのほうがライブラリのインストール無しで完結した再現性のある内容をブログに書きやすいと思ったからです。
さて、ここからが本題ですが、今回も相変わらずの内容になっていますのでその点ご承知おきください。
----------ここから本題----------
以前書いた次の記事でAWS CodePipelineの承認アクションとAmazon EventBridgeを使用してAWS Step Functionsのワークフローへ承認フローを追加する方法を試してみました。
今回はこのAWS Step Functionsの承認フローをコンポーネント化し、別のAWS Step Functionsのワークフローから呼び出して多段階承認フローを作成する方法を試してみたいと思います。
※本記事および当執筆者のその他の記事で掲載されているソースコードは自主研究活動の一貫として作成したものであり、動作を保証するものではありません。使用する場合は自己責任でお願い致します。また、予告なく修正することもありますのでご了承ください。
今回の記事の内容は次のような構成になっています。
本記事で試す構成図
今回試す構成は次のようにコンポーネント化したAWS Step Functions承認フローを3回使用する3段階の承認フローとなります。
AWS CodePipeline内部で多段階の承認フローを作成することも可能ですが、今回は3段階それぞれにパイプラインを用意して各パイプラインの承認結果がAWS Step Functionsのワークフローに反映される方式にしています。
コンポーネント化したAWS Step Functions承認フロー
コンポーネント化したAWS Step Functions承認フローは次の記事で紹介したものです。
参考： AWS Step Functionsのワークフローへ承認フローを追加する方法(AWS CodePipeline ＆ Amazon EventBridge編)
上記の記事でも紹介したように、AWS Systems Manager Automationを使用する承認フローと比較すると、AWS CodePipelineは実行前にステージの設定でSource Artifactを保存するAmazon S3バケット、オブジェクト名、承認時のメッセージ、確認用URLなどを予め決めておく必要があります。
AWS Systems Manager AutomationではSSM Documentで定義したパラメータの値を変更することで、Automationの実行ごとに承認者のIAMロール、最終承認に必要な複数の承認者の承認数、承認時のメッセージ、確認用URL、確認用ファイルなどを変更できます。
一方でAWS CodePipelineはAmazon S3オブジェクトのPUTのみで実行できますが、AWS Systems Manager Automationでは複数のパラメータや実行用IAMロールなどを指定して実行する必要があります。
そのため、注意するべき特徴としてAWS Systems Manager Automationはオートメーションの部品として承認フローを必要とするAWS Step Functionsに柔軟に導入することができますが、AWS CodePipelineは構築するCI/CDのアーキテクチャに応じて承認フローを含むパイプラインの構成やAWS CodePipelineの実行を必要とするAWS Step Functionsの構成を目的に応じて変更する必要があることが挙げられます。
AWS CloudFormationテンプレートとパラメータの例
AWS CloudFormationテンプレート(AWS Step FunctionsへのAWS Lambda、AWS CodePipeline、Amazon EventBridgeによる多段階承認フローの追加)
入力パラメータ例
EventRuleForCodePipelineResultState: ENABLED #Amazon EventBridgeの有効化(ENABLED)、無効化(DISABLED)の設定 Level1CodePipelineConfirmationCustomData: Approval request has been received. Please review file at the following URL to decide whether to approve or deny. #承認フローの確認ダイアログで表示するカスタムデータ(メッセージ) Level1CodePipelineConfirmationUrl: https://hidekazu-konishi.com/ #承認フローの確認ダイアログで表示する確認用URL Level1CodePipelineName: Level1CodePipelineApprovalSample #AWS CodePipelineの名称 Level1CodePipelineS3bucketKeyContentType text/html #Source Artifactのコンテンツタイプ Level1CodePipelineS3bucketKeyInput: index_level1.html #AWS CodePipelineを起動させるSource Artifactのファイル名 Level1CodePipelineS3bucketKeyOutput: index_level1_approved.html #AWS CodePipelineで承認後、最終的にデプロイするArtifactのファイル名 Level1CodePipelineS3bucketName: h-o2k #AWS CodePipelineでArtifactを保存するAmazon S3バケット名 Level1EmailForNotification: sample1@h-o2k.com #承認リクエストを送信するメールアドレス Level2CodePipelineConfirmationCustomData: Approval request has been received. Please review file at the following URL to decide whether to approve or deny. #承認フローの確認ダイアログで表示するカスタムデータ(メッセージ) Level2CodePipelineConfirmationUrl: https://hidekazu-konishi.com/ #承認フローの確認ダイアログで表示する確認用URL Level2CodePipelineName: Level2CodePipelineApprovalSample #AWS CodePipelineの名称 Level2CodePipelineS3bucketKeyContentType text/html #Source Artifactのコンテンツタイプ Level2CodePipelineS3bucketKeyInput: index_level2.html #AWS CodePipelineを起動させるSource Artifactのファイル名 Level2CodePipelineS3bucketKeyOutput: index_level2_approved.html #AWS CodePipelineで承認後、最終的にデプロイするArtifactのファイル名 Level2CodePipelineS3bucketName: h-o2k #AWS CodePipelineでArtifactを保存するAmazon S3バケット名 Level2EmailForNotification: sample2@h-o2k.com #承認リクエストを送信するメールアドレス Level3CodePipelineConfirmationCustomData: Approval request has been received. Please review file at the following URL to decide whether to approve or deny. #承認フローの確認ダイアログで表示するカスタムデータ(メッセージ) Level3CodePipelineConfirmationUrl: https://hidekazu-konishi.com/ #承認フローの確認ダイアログで表示する確認用URL Level3CodePipelineName: Level3CodePipelineApprovalSample #AWS CodePipelineの名称 Level3CodePipelineS3bucketKeyContentType text/html #Source Artifactのコンテンツタイプ Level3CodePipelineS3bucketKeyInput: index_level3.html #AWS CodePipelineを起動させるSource Artifactのファイル名 Level3CodePipelineS3bucketKeyOutput: index_level3_approved.html #AWS CodePipelineで承認後、最終的にデプロイするArtifactのファイル名 Level3CodePipelineS3bucketName: h-o2k #AWS CodePipelineでArtifactを保存するAmazon S3バケット名 Level3EmailForNotification: sample3@h-o2k.com #承認リクエストを送信するメールアドレス
テンプレート本体
ファイル名：SfnApprovalCFnSfnWithMultiLevelCodePipelineApprovalEventBridge.yml
実装で注意するべき点としてはコンポーネント化したAWS Step Functionsを呼び出すAWS Step FunctionsのIAMロールのポリシー権限が挙げられます。
AWS Step FunctionsからAWS Step Functionsを呼び出す場合のResource指定には次の3つがあります。
arn:aws:states:::states:startExecution：非同期arn:aws:states:::states:startExecution.sync：同期(Outputが文字列)arn:aws:states:::states:startExecution.sync:2：同期(OutputがJSON)
これらの非同期と同期では必要なIAMロールのポリシー権限が異なるため、その点に注意が必要です。
今回は承認フローを同期的に処理し、OutputをJSON形式で受け取って処理するのでstartExecution.sync:2を使用して必要なIAMロールのポリシー権限を設定しています。
また、AWS Step FunctionsからAWS Step Functionsを呼び出す場合のInputには次のパラメータを指定します。
"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$": "$$.Execution.Id"
AWSTemplateFormatVersion: '2010-09-09'
Description: 'Add AWS CodePipeline Approval Action to AWS Step Functions.'
Parameters:
EventRuleForCodePipelineResultState:
Type: String
Default: ENABLED
AllowedValues:
- ENABLED
- DISABLED
Level1EmailForNotification:
Type: String
Default: "sample1@h-o2k.com"
Level1CodePipelineName:
Type: String
Default: "Level1CodePipelineApprovalSample"
Level1CodePipelineS3bucketName:
Type: String
Default: "h-o2k"
Level1CodePipelineS3bucketKeyInput:
Type: String
Default: "index_level1.html"
Level1CodePipelineS3bucketKeyOutput:
Type: String
Default: "index_level1_approved.html"
Level1CodePipelineS3bucketKeyContentType:
Type: String
Default: "text/html"
Level1CodePipelineConfirmationCustomData:
Type: String
Default: "Approval request has been received. Please review file at the following URL to decide whether to approve or deny."
Level1CodePipelineConfirmationUrl:
Type: String
Default: "https://hidekazu-konishi.com/"
Level2EmailForNotification:
Type: String
Default: "sample2@h-o2k.com"
Level2CodePipelineName:
Type: String
Default: "Level2CodePipelineApprovalSample"
Level2CodePipelineS3bucketName:
Type: String
Default: "h-o2k"
Level2CodePipelineS3bucketKeyInput:
Type: String
Default: "index_level2.html"
Level2CodePipelineS3bucketKeyOutput:
Type: String
Default: "index_level2_approved.html"
Level2CodePipelineS3bucketKeyContentType:
Type: String
Default: "text/html"
Level2CodePipelineConfirmationCustomData:
Type: String
Default: "Approval request has been received. Please review file at the following URL to decide whether to approve or deny."
Level2CodePipelineConfirmationUrl:
Type: String
Default: "https://hidekazu-konishi.com/"
Level3EmailForNotification:
Type: String
Default: "sample3@h-o2k.com"
Level3CodePipelineName:
Type: String
Default: "Level3CodePipelineApprovalSample"
Level3CodePipelineS3bucketName:
Type: String
Default: "h-o2k"
Level3CodePipelineS3bucketKeyInput:
Type: String
Default: "index_level3.html"
Level3CodePipelineS3bucketKeyOutput:
Type: String
Default: "index_level3_approved.html"
Level3CodePipelineS3bucketKeyContentType:
Type: String
Default: "text/html"
Level3CodePipelineConfirmationCustomData:
Type: String
Default: "Approval request has been received. Please review file at the following URL to decide whether to approve or deny."
Level3CodePipelineConfirmationUrl:
Type: String
Default: "https://hidekazu-konishi.com/"
Resources:
AWSCodePipelineServiceRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub 'AWSCodePipelineServiceRole-${AWS::Region}'
Path: /
MaxSessionDuration: 43200
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- codepipeline.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: !Sub 'IAMPolicy-AWSCodePipelineServiceRole-${AWS::Region}'
PolicyDocument:
Version: "2012-10-17"
Statement:
- Action:
- "iam:PassRole"
Resource: "*"
Effect: Allow
Condition:
StringEqualsIfExists:
"iam:PassedToService":
- cloudformation.amazonaws.com
- elasticbeanstalk.amazonaws.com
- ec2.amazonaws.com
- ecs-tasks.amazonaws.com
- Action:
- "codecommit:CancelUploadArchive"
- "codecommit:GetBranch"
- "codecommit:GetCommit"
- "codecommit:GetRepository"
- "codecommit:GetUploadArchiveStatus"
- "codecommit:UploadArchive"
Resource: "*"
Effect: Allow
- Action:
- "codedeploy:CreateDeployment"
- "codedeploy:GetApplication"
- "codedeploy:GetApplicationRevision"
- "codedeploy:GetDeployment"
- "codedeploy:GetDeploymentConfig"
- "codedeploy:RegisterApplicationRevision"
Resource: "*"
Effect: Allow
- Action:
- "codestar-connections:UseConnection"
Resource: "*"
Effect: Allow
- Action:
- "elasticbeanstalk:*"
- "ec2:*"
- "elasticloadbalancing:*"
- "autoscaling:*"
- "cloudwatch:*"
- "s3:*"
- "sns:*"
- "cloudformation:*"
- "rds:*"
- "sqs:*"
- "ecs:*"
Resource: "*"
Effect: Allow
- Action:
- "lambda:InvokeFunction"
- "lambda:ListFunctions"
Resource: "*"
Effect: Allow
- Action:
- "opsworks:CreateDeployment"
- "opsworks:DescribeApps"
- "opsworks:DescribeCommands"
- "opsworks:DescribeDeployments"
- "opsworks:DescribeInstances"
- "opsworks:DescribeStacks"
- "opsworks:UpdateApp"
- "opsworks:UpdateStack"
Resource: "*"
Effect: Allow
- Action:
- "cloudformation:CreateStack"
- "cloudformation:DeleteStack"
- "cloudformation:DescribeStacks"
- "cloudformation:UpdateStack"
- "cloudformation:CreateChangeSet"
- "cloudformation:DeleteChangeSet"
- "cloudformation:DescribeChangeSet"
- "cloudformation:ExecuteChangeSet"
- "cloudformation:SetStackPolicy"
- "cloudformation:ValidateTemplate"
Resource: "*"
Effect: Allow
- Action:
- "codebuild:BatchGetBuilds"
- "codebuild:StartBuild"
- "codebuild:BatchGetBuildBatches"
- "codebuild:StartBuildBatch"
Resource: "*"
Effect: Allow
- Effect: Allow
Action:
- "devicefarm:ListProjects"
- "devicefarm:ListDevicePools"
- "devicefarm:GetRun"
- "devicefarm:GetUpload"
- "devicefarm:CreateUpload"
- "devicefarm:ScheduleRun"
Resource: "*"
- Effect: Allow
Action:
- "servicecatalog:ListProvisioningArtifacts"
- "servicecatalog:CreateProvisioningArtifact"
- "servicecatalog:DescribeProvisioningArtifact"
- "servicecatalog:DeleteProvisioningArtifact"
- "servicecatalog:UpdateProduct"
Resource: "*"
- Effect: Allow
Action:
- "cloudformation:ValidateTemplate"
Resource: "*"
- Effect: Allow
Action:
- "ecr:DescribeImages"
Resource: "*"
- Effect: Allow
Action:
- "states:DescribeExecution"
- "states:DescribeStateMachine"
- "states:StartExecution"
Resource: "*"
- Effect: Allow
Action:
- "appconfig:StartDeployment"
- "appconfig:StopDeployment"
- "appconfig:GetDeployment"
Resource: "*"
CodePipelineForApprovalActionLevel1:
DependsOn:
- AWSCodePipelineServiceRole
- SnsCodePipelineApprovalNotificationLevel1
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref Level1CodePipelineS3bucketName
Type: S3
Name: !Ref Level1CodePipelineName
RoleArn: !GetAtt AWSCodePipelineServiceRole.Arn
Stages:
- Name: Source
Actions:
- Name: Source
Region: !Ref AWS::Region
ActionTypeId:
Category: Source
Owner: AWS
Provider: S3
Version: '1'
Configuration:
S3Bucket: !Ref Level1CodePipelineS3bucketName
S3ObjectKey: !Ref Level1CodePipelineS3bucketKeyInput
OutputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: Approval
Actions:
- Name: Approval
Region: !Ref AWS::Region
ActionTypeId:
Category: Approval
Owner: AWS
Provider: Manual
Version: '1'
Configuration:
CustomData: !Ref Level1CodePipelineConfirmationCustomData
ExternalEntityLink: !Ref Level1CodePipelineConfirmationUrl
NotificationArn: !Ref SnsCodePipelineApprovalNotificationLevel1
RunOrder: 1
- Name: Deploy
Actions:
- Name: Deploy
Region: !Ref AWS::Region
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: S3
Version: '1'
Configuration:
BucketName: !Ref Level1CodePipelineS3bucketName
ObjectKey: !Ref Level1CodePipelineS3bucketKeyOutput
Extract: false
InputArtifacts:
- Name: SourceArtifact
RunOrder: 1
CodePipelineForApprovalActionLevel2:
DependsOn:
- AWSCodePipelineServiceRole
- SnsCodePipelineApprovalNotificationLevel2
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref Level2CodePipelineS3bucketName
Type: S3
Name: !Ref Level2CodePipelineName
RoleArn: !GetAtt AWSCodePipelineServiceRole.Arn
Stages:
- Name: Source
Actions:
- Name: Source
Region: !Ref AWS::Region
ActionTypeId:
Category: Source
Owner: AWS
Provider: S3
Version: '1'
Configuration:
S3Bucket: !Ref Level2CodePipelineS3bucketName
S3ObjectKey: !Ref Level2CodePipelineS3bucketKeyInput
OutputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: Approval
Actions:
- Name: Approval
Region: !Ref AWS::Region
ActionTypeId:
Category: Approval
Owner: AWS
Provider: Manual
Version: '1'
Configuration:
CustomData: !Ref Level2CodePipelineConfirmationCustomData
ExternalEntityLink: !Ref Level2CodePipelineConfirmationUrl
NotificationArn: !Ref SnsCodePipelineApprovalNotificationLevel2
RunOrder: 1
- Name: Deploy
Actions:
- Name: Deploy
Region: !Ref AWS::Region
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: S3
Version: '1'
Configuration:
BucketName: !Ref Level2CodePipelineS3bucketName
ObjectKey: !Ref Level2CodePipelineS3bucketKeyOutput
Extract: false
InputArtifacts:
- Name: SourceArtifact
RunOrder: 1
CodePipelineForApprovalActionLevel3:
DependsOn:
- AWSCodePipelineServiceRole
- SnsCodePipelineApprovalNotificationLevel3
Type: AWS::CodePipeline::Pipeline
Properties:
ArtifactStore:
Location: !Ref Level3CodePipelineS3bucketName
Type: S3
Name: !Ref Level3CodePipelineName
RoleArn: !GetAtt AWSCodePipelineServiceRole.Arn
Stages:
- Name: Source
Actions:
- Name: Source
Region: !Ref AWS::Region
ActionTypeId:
Category: Source
Owner: AWS
Provider: S3
Version: '1'
Configuration:
S3Bucket: !Ref Level3CodePipelineS3bucketName
S3ObjectKey: !Ref Level3CodePipelineS3bucketKeyInput
OutputArtifacts:
- Name: SourceArtifact
RunOrder: 1
- Name: Approval
Actions:
- Name: Approval
Region: !Ref AWS::Region
ActionTypeId:
Category: Approval
Owner: AWS
Provider: Manual
Version: '1'
Configuration:
CustomData: !Ref Level3CodePipelineConfirmationCustomData
ExternalEntityLink: !Ref Level3CodePipelineConfirmationUrl
NotificationArn: !Ref SnsCodePipelineApprovalNotificationLevel3
RunOrder: 1
- Name: Deploy
Actions:
- Name: Deploy
Region: !Ref AWS::Region
ActionTypeId:
Category: Deploy
Owner: AWS
Provider: S3
Version: '1'
Configuration:
BucketName: !Ref Level3CodePipelineS3bucketName
ObjectKey: !Ref Level3CodePipelineS3bucketKeyOutput
Extract: false
InputArtifacts:
- Name: SourceArtifact
RunOrder: 1
LambdaForCodePipelineExecution:
Type: AWS::Lambda::Function
DependsOn:
- LambdaForCodePipelineExecutionRole
Properties:
FunctionName: LambdaForCodePipelineExecution
Description : 'LambdaForCodePipelineExecution'
Runtime: python3.9
MemorySize: 10240
Timeout: 900
Role: !GetAtt LambdaForCodePipelineExecutionRole.Arn
Handler: index.lambda_handler
Code:
ZipFile: |
import botocore
import boto3
import json
import os
import sys
region = os.environ.get('AWS_REGION')
sts_client = boto3.client("sts", region_name=region)
account_id = sts_client.get_caller_identity()["Account"]
s3_client = boto3.client('s3', region_name=region)
def lambda_handler(event, context):
print(("Received event: " + json.dumps(event, indent=2)))
try:
s3_put_res = s3_client.put_object(
Body=event['confirmation_file_content'],
Bucket=event['s3_bucket_name'],
Key=event['s3_bucket_key'],
ContentType=event['confirmation_file_content-type'],
Metadata={
'x-amz-meta-sfntoken': event['token']
}
)
print('s3_client.put_object: ')
print(s3_put_res)
except Exception as ex:
print(f'Exception:{ex}')
tb = sys.exc_info()[2]
print(f's3_client put_object FAIL. Exception:{str(ex.with_traceback(tb))}')
raise
result = {}
result['params'] = event.copy()
return result
LambdaForCodePipelineExecutionRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub 'IAMRole-LambdaForCodePipelineExecutionRole-${AWS::Region}'
Path: /
MaxSessionDuration: 43200
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- edgelambda.amazonaws.com
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: !Sub 'IAMPolicy-LambdaForCodePipelineExecutionRole-${AWS::Region}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
Resource:
- 'arn:aws:logs:*:*:*'
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
Resource:
- !Sub 'arn:aws:logs:*:*:log-group:/aws/lambda/LambdaForCodePipelineExecutionRole:*'
- Effect: Allow
Action:
- s3:PutObject
Resource:
- '*'
LambdaForReceivingCodePipelineResult:
Type: AWS::Lambda::Function
DependsOn:
- LambdaForReceivingCodePipelineResultRole
Properties:
FunctionName: LambdaForReceivingCodePipelineResult
Description : 'LambdaForReceivingCodePipelineResult'
Runtime: python3.9
MemorySize: 10240
Timeout: 900
Role: !GetAtt LambdaForReceivingCodePipelineResultRole.Arn
Handler: index.lambda_handler
Code:
ZipFile: |
import botocore
import boto3
import json
import os
import sys
region = os.environ.get('AWS_REGION')
sts_client = boto3.client("sts", region_name=region)
account_id = sts_client.get_caller_identity()["Account"]
sns_client = boto3.client('sns', region_name=region)
cpl_client = boto3.client('codepipeline', region_name=region)
s3_client = boto3.client('s3', region_name=region)
sfn_client = boto3.client('stepfunctions', region_name=region)
def lambda_handler(event, context):
print(("Received event: " + json.dumps(event, indent=2)))
sfn_token = ''
is_approved = False
try:
#Eventのexecution-idからPipeline実行で使用したrevisionId(Amazon S3オブジェクトバージョン)を特定し、Step Functionsのトークンを取得する。
cpl_res_exe = cpl_client.get_pipeline_execution(
pipelineName=event['detail']['pipeline'],
pipelineExecutionId=event['detail']['execution-id']
)
print('cpl_client.get_pipeline_execution: ')
print(cpl_res_exe)
s3_version_id = cpl_res_exe['pipelineExecution']['artifactRevisions'][0]['revisionId']
print(f's3_version_id: {s3_version_id}')
cpl_res = cpl_client.get_pipeline(
name=cpl_res_exe['pipelineExecution']['pipelineName'],
version=cpl_res_exe['pipelineExecution']['pipelineVersion']
)
#パイプライン名とパイプラインバージョンから使用しているAmazon S3バケットとオブジェクトキーを取得する。
s3_bucket_name = cpl_res['pipeline']['stages'][0]['actions'][0]['configuration']['S3Bucket']
print(f's3_bucket_name: {s3_bucket_name}')
s3_bucket_key = cpl_res['pipeline']['stages'][0]['actions'][0]['configuration']['S3ObjectKey']
print(f's3_bucket_key: {s3_bucket_key}')
#バージョンIDに対応するAmazon S3オブジェクトのメタデータにあるStep Functionsのトークンを取得する。
s3_get_res = s3_client.get_object(Bucket=s3_bucket_name, Key=s3_bucket_key, VersionId=s3_version_id)
print('s3_client.get_object: ')
print(s3_get_res)
#file_content = s3_get_res['Body'].read().decode('utf-8')
sfn_token = s3_get_res['Metadata']['x-amz-meta-sfntoken']
print(f'sfn_token: {sfn_token}')
#Eventのstateから承認結果を取得する。
approval_result = event['detail'].get('state','')
print(f'approval_result:{approval_result}')
if approval_result == 'SUCCEEDED':
is_approved = True
except Exception as ex:
print(f'Exception:{ex}')
tb = sys.exc_info()[2]
print(f'cpl_client.get_pipeline_execution, s3_client.get_object FAIL. Exception:{str(ex.with_traceback(tb))}')
is_approved = False
try:
#コールバックしたトークンでSFN側にタスクの成功を送信する。
sfn_res = sfn_client.send_task_success(
taskToken=sfn_token,
output=json.dumps({'is_approved':is_approved})
)
except Exception as ex:
print(f'Exception:{ex}')
tb = sys.exc_info()[2]
print(f'sfn_client send_task_success FAIL. Exception:{str(ex.with_traceback(tb))}')
raise
return {'is_approved':is_approved}
LambdaForReceivingCodePipelineResultRole:
Type: AWS::IAM::Role
Properties:
RoleName: !Sub 'IAMRole-LambdaForReceivingCodePipelineResult-${AWS::Region}'
Path: /
MaxSessionDuration: 43200
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- edgelambda.amazonaws.com
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: !Sub 'IAMPolicy-LambdaForReceivingCodePipelineResult-${AWS::Region}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogGroup
Resource:
- 'arn:aws:logs:*:*:*'
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:PutLogEvents
Resource:
- !Sub 'arn:aws:logs:*:*:log-group:/aws/lambda/LambdaForReceivingCodePipelineResult:*'
- Effect: Allow
Action:
- 's3:GetObject*'
- codepipeline:GetPipelineExecution
- codepipeline:GetPipeline
Resource:
- '*'
- Effect: Allow
Action:
- states:ListActivities
- states:ListExecutions
- states:ListStateMachines
- states:DescribeActivity
- states:DescribeExecution
- states:DescribeStateMachine
- states:DescribeStateMachineForExecution
- states:GetExecutionHistory
- states:SendTaskSuccess
Resource:
- '*'
LambdaForReceivingCodePipelineResultPermission:
Type: AWS::Lambda::Permission
DependsOn:
- LambdaForReceivingCodePipelineResult
- EventRuleForCodePipelineResult
Properties:
Action: lambda:InvokeFunction
FunctionName: !GetAtt LambdaForReceivingCodePipelineResult.Arn
Principal: events.amazonaws.com
SourceArn: !GetAtt EventRuleForCodePipelineResult.Arn
EventRuleForCodePipelineResult:
Type: AWS::Events::Rule
DependsOn:
- LambdaForReceivingCodePipelineResult
Properties:
Name: EventRuleForCodePipelineResult
EventBusName: default
Description: 'EventRuleForCodePipelineResult'
State: !Ref EventRuleForCodePipelineResultState
EventPattern:
source:
- 'aws.codepipeline'
detail-type:
- 'CodePipeline Action Execution State Change'
detail:
pipeline:
- !Ref Level1CodePipelineName
- !Ref Level2CodePipelineName
- !Ref Level3CodePipelineName
state:
- 'SUCCEEDED'
- 'FAILED'
type:
category:
- 'Approval'
Targets:
- Id: 'EventRuleForCodePipelineResultTarget'
Arn: !GetAtt LambdaForReceivingCodePipelineResult.Arn
SnsCodePipelineApprovalNotificationLevel1:
Type: AWS::SNS::Topic
Properties:
TopicName: CodePipelineApprovalNotificationLevel1
DisplayName: CodePipelineApprovalNotificationLevel1
FifoTopic: False
Subscription:
- Endpoint: !Ref Level1EmailForNotification
Protocol: email
SnsCodePipelineApprovalNotificationLevel2:
Type: AWS::SNS::Topic
Properties:
TopicName: CodePipelineApprovalNotificationLevel2
DisplayName: CodePipelineApprovalNotificationLevel2
FifoTopic: False
Subscription:
- Endpoint: !Ref Level2EmailForNotification
Protocol: email
SnsCodePipelineApprovalNotificationLevel3:
Type: AWS::SNS::Topic
Properties:
TopicName: CodePipelineApprovalNotificationLevel3
DisplayName: CodePipelineApprovalNotificationLevel3
FifoTopic: False
Subscription:
- Endpoint: !Ref Level3EmailForNotification
Protocol: email
StepFunctionsWithCodePipelineApproval:
Type: AWS::StepFunctions::StateMachine
DependsOn:
- LambdaForCodePipelineExecution
- LambdaForReceivingCodePipelineResult
- StepFunctionsWithCodePipelineApprovalRole
- StepFunctionsWithCodePipelineApprovalLogGroup
Properties:
StateMachineName: StepFunctionsWithCodePipelineApproval
StateMachineType: STANDARD
RoleArn: !GetAtt StepFunctionsWithCodePipelineApprovalRole.Arn
LoggingConfiguration:
Level: ALL
IncludeExecutionData: true
Destinations:
- CloudWatchLogsLogGroup:
LogGroupArn: !GetAtt StepFunctionsWithCodePipelineApprovalLogGroup.Arn
DefinitionString: !Sub |-
{
"Comment": "Sample of adding an Approval flow to AWS Step Functions.",
"TimeoutSeconds": 604800,
"StartAt": "InvokeLambdaForCodePipelineExecution",
"States": {
"InvokeLambdaForCodePipelineExecution": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
"Parameters": {
"FunctionName": "${LambdaForCodePipelineExecution.Arn}:$LATEST",
"Payload": {
"step.$": "$$.State.Name",
"token.$": "$$.Task.Token",
"s3_bucket_name.$": "$$.Execution.Input.s3_bucket_name",
"s3_bucket_key.$": "$$.Execution.Input.s3_bucket_key",
"confirmation_file_content-type.$": "$$.Execution.Input.confirmation_file_content-type",
"confirmation_file_content.$": "$$.Execution.Input.confirmation_file_content"
}
},
"Retry": [
{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException",
"Lambda.TooManyRequestsException"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}
],
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"Next": "Fail"
}
],
"Next": "ApprovalResult"
},
"ApprovalResult": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.is_approved",
"BooleanEquals": true,
"Next": "Approved"
},
{
"Variable": "$.is_approved",
"BooleanEquals": false,
"Next": "Rejected"
}
],
"Default": "Rejected"
},
"Approved": {
"Type": "Succeed"
},
"Rejected": {
"Type": "Succeed"
},
"Fail": {
"Type": "Fail"
}
}
}
StepFunctionsWithCodePipelineApprovalRole:
Type: AWS::IAM::Role
DependsOn:
- LambdaForCodePipelineExecution
- LambdaForReceivingCodePipelineResult
Properties:
RoleName: !Sub 'IAMRole-StepFunctionsWithCodePipelineApproval-${AWS::Region}'
Path: /
MaxSessionDuration: 43200
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- states.amazonaws.com
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: !Sub 'IAMPolicy-StepFunctionsWithCodePipelineApproval-${AWS::Region}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- lambda:InvokeFunction
Resource:
- !Sub '${LambdaForCodePipelineExecution.Arn}:*'
- !Sub '${LambdaForReceivingCodePipelineResult.Arn}:*'
- Effect: Allow
Action:
- lambda:InvokeFunction
Resource:
- !Sub '${LambdaForCodePipelineExecution.Arn}'
- !Sub '${LambdaForReceivingCodePipelineResult.Arn}'
- PolicyName: CloudWatchLogsDeliveryFullAccessPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:DescribeResourcePolicies
- logs:DescribeLogGroups
- logs:GetLogDelivery
- logs:CreateLogDelivery
- logs:DeleteLogDelivery
- logs:UpdateLogDelivery
- logs:ListLogDeliveries
- logs:PutResourcePolicy
Resource:
- '*'
- PolicyName: XRayAccessPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- xray:PutTraceSegments
- xray:PutTelemetryRecords
- xray:GetSamplingRules
- xray:GetSamplingTargets
Resource:
- '*'
StepFunctionsWithCodePipelineApprovalLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: /aws/vendedlogs/states/Logs-StepFunctionsWithCodePipelineApproval
StepFunctionsCallerForApprovalFlow:
Type: AWS::StepFunctions::StateMachine
DependsOn:
- StepFunctionsWithCodePipelineApproval
- StepFunctionsCallerForApprovalFlowRole
- StepFunctionsCallerForApprovalFlowLogGroup
Properties:
StateMachineName: StepFunctionsCallerForApprovalFlow
StateMachineType: STANDARD
RoleArn: !GetAtt StepFunctionsCallerForApprovalFlowRole.Arn
LoggingConfiguration:
Level: ALL
IncludeExecutionData: true
Destinations:
- CloudWatchLogsLogGroup:
LogGroupArn: !GetAtt StepFunctionsCallerForApprovalFlowLogGroup.Arn
DefinitionString: !Sub |-
{
"Comment": "Sample of Upper-Level Caller for Approval Flow.",
"TimeoutSeconds": 604800,
"StartAt": "Level1ExecutionStepFunctionsWithCodePipelineApproval",
"States": {
"Level1ExecutionStepFunctionsWithCodePipelineApproval": {
"Type": "Task",
"Resource": "arn:aws:states:::states:startExecution.sync:2",
"OutputPath": "$.Output",
"Parameters": {
"StateMachineArn": "${StepFunctionsWithCodePipelineApproval.Arn}",
"Input": {
"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$": "$$.Execution.Id",
"region.$": "$$.Execution.Input.region",
"s3_bucket_name.$": "$$.Execution.Input.level1_s3_bucket_name",
"s3_bucket_key.$": "$$.Execution.Input.level1_s3_bucket_key",
"confirmation_file_content-type.$": "$$.Execution.Input.level1_confirmation_file_content-type",
"confirmation_file_content.$": "$$.Execution.Input.level1_confirmation_file_content"
}
},
"Retry": [
{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException",
"Lambda.TooManyRequestsException"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}
],
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"Next": "Fail"
}
],
"Next": "Level1ApprovalResult"
},
"Level2ExecutionStepFunctionsWithCodePipelineApproval": {
"Type": "Task",
"Resource": "arn:aws:states:::states:startExecution.sync:2",
"OutputPath": "$.Output",
"Parameters": {
"StateMachineArn": "${StepFunctionsWithCodePipelineApproval.Arn}",
"Input": {
"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$": "$$.Execution.Id",
"region.$": "$$.Execution.Input.region",
"s3_bucket_name.$": "$$.Execution.Input.level2_s3_bucket_name",
"s3_bucket_key.$": "$$.Execution.Input.level2_s3_bucket_key",
"confirmation_file_content-type.$": "$$.Execution.Input.level2_confirmation_file_content-type",
"confirmation_file_content.$": "$$.Execution.Input.level2_confirmation_file_content"
}
},
"Retry": [
{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException",
"Lambda.TooManyRequestsException"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}
],
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"Next": "Fail"
}
],
"Next": "Level2ApprovalResult"
},
"Level3ExecutionStepFunctionsWithCodePipelineApproval": {
"Type": "Task",
"Resource": "arn:aws:states:::states:startExecution.sync:2",
"OutputPath": "$.Output",
"Parameters": {
"StateMachineArn": "${StepFunctionsWithCodePipelineApproval.Arn}",
"Input": {
"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$": "$$.Execution.Id",
"region.$": "$$.Execution.Input.region",
"s3_bucket_name.$": "$$.Execution.Input.level3_s3_bucket_name",
"s3_bucket_key.$": "$$.Execution.Input.level3_s3_bucket_key",
"confirmation_file_content-type.$": "$$.Execution.Input.level3_confirmation_file_content-type",
"confirmation_file_content.$": "$$.Execution.Input.level3_confirmation_file_content"
}
},
"Retry": [
{
"ErrorEquals": [
"Lambda.ServiceException",
"Lambda.AWSLambdaException",
"Lambda.SdkClientException",
"Lambda.TooManyRequestsException"
],
"IntervalSeconds": 2,
"MaxAttempts": 6,
"BackoffRate": 2
}
],
"Catch": [
{
"ErrorEquals": [
"States.ALL"
],
"Next": "Fail"
}
],
"Next": "Level3ApprovalResult"
},
"Level1ApprovalResult": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.is_approved",
"BooleanEquals": true,
"Next": "Level2ExecutionStepFunctionsWithCodePipelineApproval"
},
{
"Variable": "$.is_approved",
"BooleanEquals": false,
"Next": "Rejected"
}
],
"Default": "Rejected"
},
"Level2ApprovalResult": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.is_approved",
"BooleanEquals": true,
"Next": "Level3ExecutionStepFunctionsWithCodePipelineApproval"
},
{
"Variable": "$.is_approved",
"BooleanEquals": false,
"Next": "Rejected"
}
],
"Default": "Rejected"
},
"Level3ApprovalResult": {
"Type": "Choice",
"Choices": [
{
"Variable": "$.is_approved",
"BooleanEquals": true,
"Next": "Approved"
},
{
"Variable": "$.is_approved",
"BooleanEquals": false,
"Next": "Rejected"
}
],
"Default": "Rejected"
},
"Approved": {
"Type": "Succeed"
},
"Rejected": {
"Type": "Succeed"
},
"Fail": {
"Type": "Fail"
}
}
}
StepFunctionsCallerForApprovalFlowRole:
Type: AWS::IAM::Role
DependsOn:
- StepFunctionsWithCodePipelineApproval
Properties:
RoleName: !Sub 'IAMRole-StepFunctionsCallerForApprovalFlow-${AWS::Region}'
Path: /
MaxSessionDuration: 43200
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service:
- states.amazonaws.com
- lambda.amazonaws.com
Action:
- sts:AssumeRole
Policies:
- PolicyName: !Sub 'IAMPolicy-StepFunctionsCallerForApprovalFlow-${AWS::Region}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- states:StartExecution
Resource:
- !Sub 'arn:aws:states:${AWS::Region}:${AWS::AccountId}:stateMachine:${StepFunctionsWithCodePipelineApproval.Name}'
- Effect: Allow
Action:
- states:DescribeExecution
- states:StopExecution
Resource:
- !Sub 'arn:aws:states:${AWS::Region}:${AWS::AccountId}:execution:${StepFunctionsWithCodePipelineApproval.Name}:*'
- Effect: Allow
Action:
- events:PutTargets
- events:PutRule
- events:DescribeRule
Resource:
- !Sub 'arn:aws:events:${AWS::Region}:${AWS::AccountId}:rule/StepFunctionsGetEventsForStepFunctionsExecutionRule'
- PolicyName: CloudWatchLogsDeliveryFullAccessPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:DescribeResourcePolicies
- logs:DescribeLogGroups
- logs:GetLogDelivery
- logs:CreateLogDelivery
- logs:DeleteLogDelivery
- logs:UpdateLogDelivery
- logs:ListLogDeliveries
- logs:PutResourcePolicy
Resource:
- '*'
- PolicyName: XRayAccessPolicy
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- xray:PutTraceSegments
- xray:PutTelemetryRecords
- xray:GetSamplingRules
- xray:GetSamplingTargets
Resource:
- '*'
StepFunctionsCallerForApprovalFlowLogGroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: /aws/vendedlogs/states/Logs-StepFunctionsCallerForApprovalFlow
Outputs:
Region:
Value:
!Ref AWS::Region
StepFunctionsInputExample:
Description: "AWS Step Functions Input Example"
Value: !Sub |-
{
"region": "${AWS::Region}",
"level1_s3_bucket_name": "${Level1CodePipelineS3bucketName}",
"level1_s3_bucket_key": "${Level1CodePipelineS3bucketKeyInput}",
"level1_confirmation_file_content-type": "${Level1CodePipelineS3bucketKeyContentType}",
"level1_confirmation_file_content": "<!DOCTYPE html><html><head><meta http-equiv=\"refresh\" content=\"10; URL=https:\/\/hidekazu-konishi.com\/\"><title>Demo of adding approval actions with AWS CodePipeline to AWS Step Functions<\/title><\/head><body>Demo of adding approval actions with AWS CodePipeline to AWS Step Functions.<br\/><\/body><\/html>",
"level2_s3_bucket_name": "${Level2CodePipelineS3bucketName}",
"level2_s3_bucket_key": "${Level2CodePipelineS3bucketKeyInput}",
"level2_confirmation_file_content-type": "${Level2CodePipelineS3bucketKeyContentType}",
"level2_confirmation_file_content": "<!DOCTYPE html><html><head><meta http-equiv=\"refresh\" content=\"10; URL=https:\/\/hidekazu-konishi.com\/\"><title>Demo of adding approval actions with AWS CodePipeline to AWS Step Functions<\/title><\/head><body>Demo of adding approval actions with AWS CodePipeline to AWS Step Functions.<br\/><\/body><\/html>",
"level3_s3_bucket_name": "${Level3CodePipelineS3bucketName}",
"level3_s3_bucket_key": "${Level3CodePipelineS3bucketKeyInput}",
"level3_confirmation_file_content-type": "${Level3CodePipelineS3bucketKeyContentType}",
"level3_confirmation_file_content": "<!DOCTYPE html><html><head><meta http-equiv=\"refresh\" content=\"10; URL=https:\/\/hidekazu-konishi.com\/\"><title>Demo of adding approval actions with AWS CodePipeline to AWS Step Functions<\/title><\/head><body>Demo of adding approval actions with AWS CodePipeline to AWS Step Functions.<br\/><\/body><\/html>"
}
構築手順
- AWS Step FunctionsやAWS CodePipelineをサポートしているリージョンで、テンプレートのパラメータに必要な値を入力してAWS CloudFormationでデプロイする。
AWS CloudFormationスタック作成後にOutputフィールドへAWS Step Functions実行時の入力パラメータ例(JSON形式)がStepFunctionsInputExampleとして出力されるのでメモしておく。 - 入力した各承認段階のEmailアドレスにSNSトピックのサブスクリプション承認リクエストが届くので承認しておく。
デモの実行
上記「構築手順」でメモした
StepFunctionsInputExampleのJSONパラメータのうち、levelX_confirmation_file_contentを修正し、AWS Step FunctionsステートマシンStepFunctionsCallerForApprovalFlowの入力値にして実行する。
levelX_confirmation_file_contentはデモ用として用意したAWS CodePipelineのSource Artifactのファイル内容です。実際にAWS CodePipelineを使用してアーティファクトを各AWSリソースにデプロイする場合にはZipファイルなどを使用するため、AWS Step FunctionsのステップやAWS CodePipelineのステージは用途に合わせて構築する必要があります。
※「X」には各段階の数値が入ります。構築時に指定したEmailアドレスにAWS CodePipeline承認アクションのメールが届くので、承認(
Approve)するか拒否(Reject)するかをAWSマネジメントコンソールから選択する。- AWS Step Functionsステートマシン
StepFunctionsWithCodePipelineApprovalのステップが選択した承認(Approve)、拒否(Reject)の通りに遷移することを確認する。 - 上記「2.」～「3.」を各承認段階分実行する。
各承認段階のいずれかで拒否(Reject)を選択した場合はそれが最終的な拒否の結果となる。
各承認段階のすべてで承認(Approve)を選択した場合はそれが最終的な承認の結果となる。
削除手順
- 「構築手順」で作成したAWS CloudFormationスタックを削除する。
参考：
Manage AWS Step Functions Executions as an Integrated Service - AWS Step Functions
IAM Policies for integrated services:AWS Step Functions - AWS Step Functions
Tech Blog with related articles referenced
まとめ
今回はAWS CodePipelineの承認アクションをコンポーネント化し、別のAWS Step Functionsのワークフローから呼び出す多段階承認フローを作成する方法を試しました。
機会があれば今まで紹介したAWSサービスを使用した承認フローについて各パターンの特徴をまとめたいと思います。
Written by Hidekazu Konishi
