1. 首页
  2. 问答
  3. 如何处理CodePipeline中的多个环境？

如何处理CodePipeline中的多个环境？

2017-01-16 75 views
2

我正在使用代码管道来部署我的基础架构，我希望能够在不同的环境（dev，staging，prod，...）中部署它。如何处理CodePipeline中的多个环境？

我现在有一些含有“PIP安装”的说明和“AWS cloudformation包”命令buildspec.yml文件。我还创建了2个管道，一个用于生产，另一个用于开发，指向github上的2个不同分支。我遇到的问题是，因为在两个分支中文件都包含相似的资源，所以我在S3存储桶上存在名称冲突。

在使用AWS CLI和cloudformation创建或更新一个堆栈，你可以通过使用--parameters选项参数。我想在我创建的2个管道中做类似的事情。

什么是解决这一问题的最佳解决方案？

的最终目标是自动化我们的基础设施的部署。我们的基础架构由用户，KMS密钥，Lamdbas（在Python中），组和存储桶组成。

我已创建的教程以下两条管线：http://docs.aws.amazon.com/lambda/latest/dg/automating-deployment.html

第一管道被连接到包含代码，第二个到临时分支回购的主分支。我的目标是使用第二个管道环境中的第一个管道和临时分支在临时环境中自动部署生产环境中的主分支。

我buildspec.yml文件的样子：

version: 0.1 
phases: 
    install: 
     commands: 
      - pip install requests -t . 
      - pip install simplejson -t . 
      - pip install Image -t . 
      - aws cloudformation package --template-file image_processing_sam.yml --s3-bucket package-bucket --output-template-file new_image_processing_sam.yml 
artifacts: 
    type: zip 
    files: 
     - new_image_processing_sam.yml

的image_processing_sam.yml文件的样子：

AWSTemplateFormatVersion: "2010-09-09" 
Transform: "AWS::Serverless-2016-10-31" 
Description: Create a thumbnail for an image uploaded to S3 
Resources: 

    ThumbnailFunction: 
    Type: "AWS::Serverless::Function" 
    Properties: 
     Role: !GetAtt LambdaExecutionRole.Arn 
     Handler: create_thumbnail.handler 
     Runtime: python2.7 
     Timeout: 30 
     Description: "A function computing the thumbnail for an image." 

    LambdaSecretEncryptionKey: 
    Type: "AWS::KMS::Key" 
    Properties: 
     Description: "A key used to encrypt secrets used in the Lambda functions" 
     Enabled: True 
     EnableKeyRotation: False 
     KeyPolicy: 
     Version: "2012-10-17" 
     Id: "lambda-secret-encryption-key" 
     Statement: 
      - 
      Sid: "Allow administration of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role" 
      Action: 
       - "kms:Create*" 
       - "kms:Describe*" 
       - "kms:Enable*" 
       - "kms:List*" 
       - "kms:Put*" 
       - "kms:Update*" 
       - "kms:Revoke*" 
       - "kms:Disable*" 
       - "kms:Get*" 
       - "kms:Delete*" 
       - "kms:ScheduleKeyDeletion" 
       - "kms:CancelKeyDeletion" 
      Resource: "*" 
      - 
      Sid: "Allow use of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: 
       - !GetAtt LambdaExecutionRole.Arn 
      Action: 
       - "kms:Encrypt" 
       - "kms:Decrypt" 
       - "kms:ReEncrypt*" 
       - "kms:GenerateDataKey*" 
       - "kms:DescribeKey" 
      Resource: "*" 

    LambdaExecutionRole: 
    Type: "AWS::IAM::Role" 
    Properties: 
     RoleName: "LambdaExecutionRole" 
     AssumeRolePolicyDocument: 
     Version: "2012-10-17" 
     Statement: 
     - Effect: Allow 
      Principal: 
      Service: 
      - "lambda.amazonaws.com" 
      Action: 
      - "sts:AssumeRole" 
     Policies: 
     - 
      PolicyName: LambdaKMS 
      PolicyDocument: 
      Version: "2012-10-17" 
      Statement: 
       - 
       Effect: "Allow" 
       Action: 
        - "kms:Decrypt" 
       Resource: "*" 
       - 
       Effect: "Allow" 
       Action: 
        - "lambda:InvokeFunction" 
       Resource: "*" 
     ManagedPolicyArns: 
     - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 

    UserGroup: 
     Type: "AWS::IAM::Group" 

    LambdaTriggerUser: 
    Type: "AWS::IAM::User" 
    Properties: 
     UserName: "LambdaTriggerUser" 

    LambdaTriggerUserKeys: 
    Type: "AWS::IAM::AccessKey" 
    Properties: 
     UserName: 
     Ref: LambdaTriggerUser 

    Users: 
    Type: "AWS::IAM::UserToGroupAddition" 
    Properties: 
     GroupName: 
     Ref: UserGroup 
     Users: 
     - Ref: LambdaTriggerUser 

    Policies: 
    Type: "AWS::IAM::Policy" 
    Properties: 
     PolicyName: UserPolicy 
     PolicyDocument: 
     Statement: 
      - 
      Effect: "Allow" 
      Action: 
       - "lambda:InvokeFunction" 
      Resource: 
       - !GetAtt DispatcherFunction.Arn 
     Groups: 
     - Ref: UserGroup 

    PackageBucket: 
    Type: "AWS::S3::Bucket" 
    Properties: 
     BucketName: "package-bucket" 
     VersioningConfiguration: 
     Status: "Enabled" 

Outputs: 
    LambdaTriggerUserAccessKey: 
    Value: 
     Ref: "LambdaTriggerUserKeys" 
    Description: "AWSAccessKeyId of LambdaTriggerUser" 

    LambdaTriggerUserSecretKey: 
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey 
    Description: "AWSSecretKey of LambdaTriggerUser"

我已在两条流水线部署动作来执行计算的变更集在测试行动期间。

第一管线工程就像一个魅力和做的一切我期望它做的事。每次我在主分支中推送代码时，都会部署它。

我现在面临的问题是，当我在临时分支推代码，一切都在流水线工作，直到达到部署行动。部署操作尝试创建一个新的堆栈，但由于它与处理的buildspec.yml和image_processing_sam.yml完全相同，因此我会遇到下面的名称冲突。

package-bucket already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx 
LambdaTriggerUser already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx 
LambdaExecutionRole already exists in stack arn:aws:cloudformation:eu-west-1:xxxxxxxxxxxx:stack/master/xxxxxx-xxxx-xxx-xxxx 
...

有没有办法参数化buildspec.yml能够添加一个后缀在image_processing_sam.yml资源的名字吗？任何其他想法实现这一点是值得欢迎的。

此致敬礼。

来源

2017-01-16 JonathanGailliez

回答

2

http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html

模板配置文件通过参数文件应用到CodePipeline的CloudFormation这样的：

{ 
    "Parameters" : { 
    "DBName" : "TestWordPressDB", 
    "DBPassword" : "TestDBRootPassword", 
    "DBRootPassword" : "TestDBRootPassword", 
    "DBUser" : "TestDBuser",  
    "KeyName" : "TestEC2KeyName" 
    } 
}

将这些文件在您的回购协议的根源，他们可以被引用在至少有两种方法。

在你CodePipeline CloudFormation：

Configuration: 
    ActionMode: REPLACE_ON_FAILURE 
    RoleArn: !GetAtt [CFNRole, Arn] 
    StackName: !Ref TestStackName 
    TemplateConfiguration: !Sub "TemplateSource::${TestStackConfig}" 
    TemplatePath: !Sub "TemplateSource::${TemplateFileName}"

或者在模板配置字段控制台： enter image description here

值得注意的是，该配置文件的格式是通过CLI从CloudFormation不同的使用

-- parameters

- 参数使用此格式：

[ 
    { 
    "ParameterKey": "team", 
    "ParameterValue": "AD-Student Life Applications" 
    }, 
    { 
    "ParameterKey": "env", 
    "ParameterValue": "dev" 
    }, 
    { 
    "ParameterKey": "dataSensitivity", 
    "ParameterValue": "public" 
    }, 
    { 
    "ParameterKey": "app", 
    "ParameterValue": "events-list-test" 
    } 
]

CodePipeline Cloudformation模板配置文件，请使用以下格式：

{ 
    "Parameters" : { 
    "DBName" : "TestWordPressDB", 
    "DBPassword" : "TestDBRootPassword", 
    "DBRootPassword" : "TestDBRootPassword", 
    "DBUser" : "TestDBuser",  
    "KeyName" : "TestEC2KeyName" 
    } 
}

来源

2017-01-19 17:22:29

+1

您好，我没有这个时间这个星期来测试，但我一定要尝试一下，尽快接受你的答案。感谢您的时间和详细的答案。 – JonathanGailliez

+0

昨天测试完整代码示例可在此处获得第271行： https://github.com/byu-oit-appdev/iac/blob/master/cloudformation/codepipeline/lambda-pipeline-cf.yaml –

+1

嗨，Eric，我终于设法花时间尝试和验证您的解决方案。这确实是继续下去的方式。再次感谢您的详细解答和您的时间。最好的祝福。 – JonathanGailliez

1

检查Eric Nord的答案。这是你正在寻找的人。

我问了AWS论坛上的问题以及here。

这里是由AWS提供的解决方案：

嗨，

如果你的目标是有分期和掌握不同桶的名字，然后又选择是使用CloudFormation参数。

编辑现有管道时，如果编辑操作，则可以展开“高级”面板并输入参数覆盖以为每个阶段指定不同的存储桶前缀。您还可以在工件中将参数作为单独的.json文件输入。

有做的是，这里更多的细节：http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-parameter-override-functions.html

以下是带有测试和生产不同的堆栈配置的全穿行：http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/continuous-delivery-codepipeline-basic-walkthrough.html

  • 蒂姆。

你一定要按照提供的文件。以下是我在下面提出的解决方案。

这是我自己的解决方案，我不满意。

我已经添加了一个在构建时运行的脚本，并修改了我的模板，给出了构建项目的CodeBuild代理的ARN。

我添加了“BRANCH_NAME”，其中可能会发生命名冲突。该image_processing_sam.yml现在是：

AWSTemplateFormatVersion: "2010-09-09" 
Transform: "AWS::Serverless-2016-10-31" 
Description: Create a thumbnail for an image uploaded to S3 
Resources: 

    ThumbnailFunction: 
    Type: "AWS::Serverless::Function" 
    Properties: 
     Role: !GetAtt LambdaExecutionRole.Arn 
     Handler: create_thumbnail.handler 
     Runtime: python2.7 
     Timeout: 30 
     Description: "A function computing the thumbnail for an image." 

    LambdaSecretEncryptionKey: 
    Type: "AWS::KMS::Key" 
    Properties: 
     Description: "A key used to encrypt secrets used in the Lambda functions" 
     Enabled: True 
     EnableKeyRotation: False 
     KeyPolicy: 
     Version: "2012-10-17" 
     Id: "lambda-secret-encryption-keyBRANCH_NAME" 
     Statement: 
      - 
      Sid: "Allow administration of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: "arn:aws:iam::xxxxxxxxxxxxx:role/cloudformation-lambda-execution-role" 
      Action: 
       - "kms:Create*" 
       - "kms:Describe*" 
       - "kms:Enable*" 
       - "kms:List*" 
       - "kms:Put*" 
       - "kms:Update*" 
       - "kms:Revoke*" 
       - "kms:Disable*" 
       - "kms:Get*" 
       - "kms:Delete*" 
       - "kms:ScheduleKeyDeletion" 
       - "kms:CancelKeyDeletion" 
      Resource: "*" 
      - 
      Sid: "Allow use of the key" 
      Effect: "Allow" 
      Principal: 
       AWS: 
       - !GetAtt LambdaExecutionRole.Arn 
      Action: 
       - "kms:Encrypt" 
       - "kms:Decrypt" 
       - "kms:ReEncrypt*" 
       - "kms:GenerateDataKey*" 
       - "kms:DescribeKey" 
      Resource: "*" 

    LambdaExecutionRole: 
    Type: "AWS::IAM::Role" 
    Properties: 
     RoleName: "LambdaExecutionRoleBRANCH_NAME" 
     AssumeRolePolicyDocument: 
     Version: "2012-10-17" 
     Statement: 
     - Effect: Allow 
      Principal: 
      Service: 
      - "lambda.amazonaws.com" 
      Action: 
      - "sts:AssumeRole" 
     Policies: 
     - 
      PolicyName: LambdaKMSBRANCH_NAME 
      PolicyDocument: 
      Version: "2012-10-17" 
      Statement: 
       - 
       Effect: "Allow" 
       Action: 
        - "kms:Decrypt" 
       Resource: "*" 
       - 
       Effect: "Allow" 
       Action: 
        - "lambda:InvokeFunction" 
       Resource: "*" 
     ManagedPolicyArns: 
     - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole" 

    UserGroup: 
     Type: "AWS::IAM::Group" 

    LambdaTriggerUser: 
    Type: "AWS::IAM::User" 
    Properties: 
     UserName: "LambdaTriggerUserBRANCH_NAME" 

    LambdaTriggerUserKeys: 
    Type: "AWS::IAM::AccessKey" 
    Properties: 
     UserName: 
     Ref: LambdaTriggerUser 

    Users: 
    Type: "AWS::IAM::UserToGroupAddition" 
    Properties: 
     GroupName: 
     Ref: UserGroup 
     Users: 
     - Ref: LambdaTriggerUser 

    Policies: 
    Type: "AWS::IAM::Policy" 
    Properties: 
     PolicyName: UserPolicyBRANCH_NAME 
     PolicyDocument: 
     Statement: 
      - 
      Effect: "Allow" 
      Action: 
       - "lambda:InvokeFunction" 
      Resource: 
       - !GetAtt DispatcherFunction.Arn 
     Groups: 
     - Ref: UserGroup 

    PackageBucket: 
    Type: "AWS::S3::Bucket" 
    Properties: 
     BucketName: "package-bucketBRANCH_NAME" 
     VersioningConfiguration: 
     Status: "Enabled" 

Outputs: 
    LambdaTriggerUserAccessKey: 
    Value: 
     Ref: "LambdaTriggerUserKeys" 
    Description: "AWSAccessKeyId of LambdaTriggerUser" 

    LambdaTriggerUserSecretKey: 
    Value: !GetAtt LambdaTriggerUserKeys.SecretAccessKey 
    Description: "AWSSecretKey of LambdaTriggerUser"

的script.sh模板替换 “BRANCH_NAME” 是：

#!/bin/bash 
echo $CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN 
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"master"* ]]; then 
    sed "s/BRANCH_NAME//g" image_processing_sam.yml > generated_image_processing_sam.yml; 
fi 
if [[ "$CODEBUILD_AGENT_ENV_CODEBUILD_BUILD_ARN" == *"staging"* ]]; then 
    sed "s/BRANCH_NAME/staging/g" image_processing_sam.yml > generated_image_processing_sam.yml; 
fi

的buildspec.yml现在是：

version: 0.1 
phases: 
    install: 
     commands: 
      # Install required module for python 
      - pip install requests -t . 
      - pip install simplejson -t . 
      - pip install Image -t . 
      - bash ./script.sh 
      # To be able to see any issue in the generated template 
      - cat generated_image_processing_sam.yml 
      # Package the generated cloudformation template in order to deploy 
      - aws cloudformation package --template-file generated_image_processing_sam.yml --s3-bucket piximate-package-bucket --output-template-file new_image_processing_sam.yml 
artifacts: 
    type: zip 
    files: 
     - new_image_processing_sam.yml

我希望它能以某种方式帮助你。如果有人能提供任何可以帮助的改进或文档，我会很高兴。

来源

2017-01-16 15:30:51 JonathanGailliez

相关问题

 相关问题