AWS CodePipeline
User Guide (API Version 2015-07-09)

The procedures in this guide support the new console design. If you choose to use the older version of the console, you will find many of the concepts and basic procedures in this guide still apply. To access help in the new console, choose the information icon.

Edit Pipelines to Use Push Events

AWS CodePipeline supports full, end-to-end continuous delivery, which includes starting your pipeline whenever there is a code change. There are two supported ways to start your pipeline upon a code change:

  • Events (either Amazon CloudWatch Events or webhooks)

  • Polling (checking periodically)

Initially, only polling was supported. Events are now the default and recommended way to start your pipeline when there’s a code change.

There are some important advantages to using push events instead of polling:

  • On average, events are significantly faster. Events should start your pipeline almost immediately, as opposed to polling, which requires waiting for the next periodic check.

  • Higher limits. Compared to pipelines that poll for changes, AWS CodePipeline can support far more event-based pipelines.

  • Better experience with many pipelines. Some customers might experience throttling or higher costs by having many pipelines continuously polling their repository for code changes. You can avoid this by using events.

When you use the AWS CodePipeline console or AWS CodeStar to create a pipeline, events are enabled by default. For backward compatibility, new pipelines created through the API, AWS CLI, or AWS CloudFormation use the original polling functionality. We strongly recommend that you use events instead. To opt in, use the AWS CLI or AWS CloudFormation to create the CloudWatch event or webhook and disable polling. Use the instructions in the following table.

You should also use events on pipelines that were created before the new console was launched. To opt in, use the AWS CodePipeline console to create the CloudWatch event or webhook and disable polling. Use the instructions in the following table .

To update AWS CodeCommit source actions

Type Action Required Instructions
If you create and manage your pipeline with the AWS CLI Use the AWS CLI to disable periodic checks and create your Amazon CloudWatch Events resources Update AWS CodeCommit Pipelines for Push Events (CLI)
If you create and manage your pipeline with AWS CloudFormation Use AWS CloudFormation to execute a change set that disables periodic checks and creates your Amazon CloudWatch Events resources Update AWS CodeCommit Pipelines for Push Events (AWS CloudFormation Template)
If you created your pipeline in the console before October 11, 2017 Use the AWS CodePipeline console to let AWS CodePipeline disable periodic checks and create your Amazon CloudWatch Events resources Update AWS CodeCommit or Amazon S3 Pipelines for Push Events (Console)
If you created your pipeline in the console after October 11, 2017 No action required

To update Amazon S3 source actions

Type Action Required Instructions
If you create and manage your pipeline with the AWS CLI Use the AWS CLI to disable periodic checks and create your Amazon CloudWatch Events and CloudTrail resources Update Amazon S3 Pipelines for Push Events (CLI)
If you create and manage your pipeline with AWS CloudFormation Use AWS CloudFormation to execute a change set that disables periodic checks and creates your Amazon CloudWatch Events and AWS CloudTrail resources Update Amazon S3 Pipelines for Push Events (AWS CloudFormation Template)
If you created your pipeline in the console before March 22, 2018 Use the AWS CodePipeline console to let AWS CodePipeline disable periodic checks and create your Amazon CloudWatch Events and AWS CloudTrail resources Update AWS CodeCommit or Amazon S3 Pipelines for Push Events (Console)
If you create your pipeline in the console after March 22, 2018 No action required

To update GitHub source actions

Type Action Required Instructions
If you create and manage your pipeline with the AWS CLI Use the AWS CLI to disable periodic checks and create and register your webhook Update GitHub Pipelines for Push Events (CLI)
If you create and manage your pipeline AWS CloudFormation Use AWS CloudFormation to execute a change set that disables periodic checks and creates and registers your webhook Update GitHub Pipelines for Push Events (AWS CloudFormation Template)
If you created your pipeline in the console before May 1, 2018 Use the AWS CodePipeline console to let AWS CodePipeline disable periodic checks and create and register your webhook Update GitHub Pipelines for Push Events (Console)
If you create your pipeline in the console after May 1, 2018 No action required

Update Pipelines for Push Events (Console)

You can use the AWS CodePipeline console to update your pipeline to use the recommended change detection method.

Update AWS CodeCommit or Amazon S3 Pipelines for Push Events (Console)

You can use the AWS CodePipeline console to update your pipeline to use Amazon CloudWatch Events to detect changes in your AWS CodeCommit source repository or your Amazon S3 source bucket.

Note

When you use the console to edit a pipeline that has an AWS CodeCommit source repository or an Amazon S3 source bucket, the rule and IAM role are created for you. If you use the AWS CLI to edit the pipeline, you must create the Amazon CloudWatch Events rule and IAM role yourself. For more information, see Use CloudWatch Events to Start an AWS CodeCommit Pipeline.

Use these steps to edit a pipeline that is using periodic checks. If you want to create a pipeline, see Create a Pipeline in AWS CodePipeline.

To edit the pipeline source stage

  1. Sign in to the AWS Management Console and open the AWS CodePipeline console at http://console.aws.amazon.com/codesuite/codepipeline/home.

    The names of all pipelines associated with your AWS account are displayed.

  2. In Name, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.

  3. On the pipeline details page, choose Edit.

  4. In Edit stage, choose the edit icon on the source action.

  5. Expand Change Detection Options and choose Use CloudWatch Events to automatically start my pipeline when a change occurs (recommended).

    A message appears showing the Amazon CloudWatch Events rule to be created for this pipeline. Choose Update.

    
                            Use the console to update an AWS CodeCommit pipeline for change
                                detection

    If you are updating an Amazon S3 pipeline, you see the following message. Choose Update.

    
                            Use the console to update an Amazon S3 pipeline for change
                                detection
  6. When you have finished editing your pipeline, choose Save pipeline changes to return to the summary page.

    A message displays the name of the Amazon CloudWatch Events rule to be created for your pipeline. Choose Save and continue.

  7. To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.

Update GitHub Pipelines for Push Events (Console)

You can use the AWS CodePipeline console to update your pipeline to use webhooks to detect changes in your AWS CodeCommit source repository.

Follow these steps to edit a pipeline that is using polling (periodic checks) to use Amazon CloudWatch Events instead. If you want to create a pipeline, see Create a Pipeline in AWS CodePipeline.

When you use the console, the PollForSourceChanges parameter for your pipelined is changed for you. The GitHub webhook is created and registered for you.

To edit the pipeline source stage

  1. Sign in to the AWS Management Console and open the AWS CodePipeline console at http://console.aws.amazon.com/codesuite/codepipeline/home.

    The names of all pipelines associated with your AWS account are displayed.

  2. In Name, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.

  3. On the pipeline details page, choose Edit.

  4. In Edit stage, choose the edit icon on the source action.

  5. Expand Change detection options and choose Use Amazon CloudWatch Events to automatically start my pipeline when a change occurs (recommended).

    A message is displayed to advise that AWS CodePipeline creates a webhook in GitHub to detect source changes. Choose Update. In addtion to the webhook, AWS CodePipeline creates the following:

    • A secret, randomly generated and used to authorize the connection to GitHub.

    • The webhook URL, generated using the public endpoint for the region.

    AWS CodePipeline registers the webhook with GitHub. This subscribes the URL to receive repository events.

    
                            Use the console to update a GitHub pipeline for change
                                detection
  6. When you have finished editing your pipeline, choose Save pipeline changes to return to the summary page.

    A message displays the name of the webhook to be created for your pipeline. Choose Save and continue.

  7. To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.

Update Pipelines for Push Events (CLI)

You can use the CLI to update your pipeline to use the recommended change detection method.

Update AWS CodeCommit Pipelines for Push Events (CLI)

Follow these steps to edit a pipeline that is using polling (periodic checks) to use a CloudWatch Events rule to start the pipeline. If you want to create a pipeline, see Create a Pipeline in AWS CodePipeline.

To build an event-driven pipeline with AWS CodeCommit, you edit the PollForSourceChanges parameter of your pipeline and then create the following resources:

  • Amazon CloudWatch Events event

  • IAM role to allow this event to start your pipeline

To edit your pipeline's PollForSourceChanges parameter

  1. Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named MyFirstPipeline, run the following command:

    aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

    This command returns nothing, but the file you created should appear in the directory where you ran the command.

  2. Open the JSON file in any plain-text editor and edit the source stage by changing the PollForSourceChanges parameter to false, as shown in this example.

    Why am I making this change? Changing this parameter to false turns off periodic checks so you can use event-based change detection only.

    "configuration": { "PollForSourceChanges": "false", "BranchName": "master", "RepositoryName": "MyTestRepo" },
  3. If you are working with the pipeline structure retrieved using the get-pipeline command, remove the metadata lines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the "metadata": { } lines and the "created", "pipelineARN", and "updated" fields.

    For example, remove the following lines from the structure:

    "metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" }

    Save the file.

  4. To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:

    Important

    Be sure to include file:// before the file name. It is required in this command.

    aws codepipeline update-pipeline --cli-input-json file://pipeline.json

    This command returns the entire structure of the edited pipeline.

    Note

    The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.

To create a CloudWatch Events rule with AWS CodeCommit as the event source and AWS CodePipeline as the target

  1. Add permissions for Amazon CloudWatch Events to use AWS CodePipeline to invoke the rule. For more information, see Using Resource-Based Policies for Amazon CloudWatch Events.

    1. Use the following sample to create the trust policy that allows CloudWatch Events to assume the service role. Name the trust policy trustpolicyforCWE.json.

      { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
    2. Use the following command to create the Role-for-MyRule role and attach the trust policy.

      aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforCWE.json
    3. Create the permissions policy JSON, as shown in this sample, for the pipeline named MyFirstPipeline. Name the permissions policy permissionspolicyforCWE.json.

      { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:80398EXAMPLE:MyFirstPipeline" ] } ] }
    4. Use the following command to attach the CodePipeline-Permissions-Policy-for-CWE permissions policy to the Role-for-MyRule role.

      Why am I making this change? Adding this policy to the role creates permissions for CloudWatch Events.

      aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-CWE --policy-document file://permissionspolicyforCWE.json
  2. Call the put-rule command and include the --name and --event-pattern parameters.

    Why am I making this change? This command enables AWS CloudFormation to create the event.

    The following sample command uses --event-pattern to create a rule called MyCodeCommitRepoRule.

    aws events put-rule --name "MyCodeCommitRepoRule" --event-pattern "{\"source\":[\"aws.codecommit\"],\"detail-type\":[\"CodeCommit Repository State Change\"],\"resources\":[\"pipeline-ARN\"],\"detail\":{\"referenceType\":[\"branch\"],\"referenceName \":[\"master\"]}}"
  3. To add AWS CodePipeline as a target, call the put-targets command and include the following parameters:

    • The --rule parameter is used with the rule_name you created by using put-rule.

    • The --targets parameter is used with the list Id of the target in the list of targets and the ARN of the target pipeline.

    The following sample command specifies that for the rule called MyCodeCommitRepoRule, the target Id is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The sample command also specifies an example ARN for the pipeline. The pipeline starts when something changes in the repository.

    aws events put-targets --rule MyCodeCommitRepoRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline

Update Amazon S3 Pipelines for Push Events (CLI)

Follow these steps to edit a pipeline that is using polling (periodic checks) to use an event in CloudWatch Events instead. If you want to create a pipeline, see Create a Pipeline in AWS CodePipeline.

To build an event-driven pipeline with Amazon S3, you edit the PollForSourceChanges parameter of your pipeline and then create the following resources:

  • AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the events.

  • Amazon CloudWatch Events event

  • IAM role to allow the CloudWatch event to start your pipeline

To create an AWS CloudTrail trail and enable logging

To use the AWS CLI to create a trail, call the create-trail command, specifying:

  • The trail name.

  • The bucket to which you have already applied the bucket policy for AWS CloudTrail.

For more information, see Creating a Trail with the AWS Command Line Interface.

  1. Call the create-trail command and include the --name and --s3-bucket-name parameters.

    Why am I making this change? This creates the CloudTrail trail required for your S3 source bucket.

    The following command uses --name and --s3-bucket-name to create a trail named my-trail and a bucket named myBucket.

    aws cloudtrail create-trail --name my-trail --s3-bucket-name myBucket
  2. Call the start-logging command and include the --name parameter.

    Why am I making this change? This command starts the CloudTrail logging for your source bucket and sends events to CloudWatch Events.

    Example:

    The following command uses --name to start logging on a trail named my-trail.

    aws cloudtrail start-logging --name my-trail
  3. Call the put-event-selectors command and include the --trail-name and --event-selectors parameters. Use event selectors to specify that you want your trail to log data events for your source bucket and send the events to the Amazon CloudWatch Events rule.

    Why am I making this change? This command filters events.

    Example:

    The following command uses --trail-name and --event-selectors to specify data events for a source bucket and prefix named myBucket/myFolder.

    aws cloudtrail put-event-selectors --trail-name my-trail --event-selectors '[{ "ReadWriteType": "WriteOnly", "IncludeManagementEvents":false, "DataResources": [{ "Type": "AWS::S3::Object", "Values": ["arn:aws:s3:::myBucket/myFolder/file.zip"] }] }]'

To create a CloudWatch Events rule with Amazon S3 as the event source and AWS CodePipeline as the target and apply the permissions policy

  1. Grant permissions for Amazon CloudWatch Events to use AWS CodePipeline to invoke the rule. For more information, see Using Resource-Based Policies for Amazon CloudWatch Events.

    1. Use the following sample to create the trust policy to allow CloudWatch Events to assume the service role. Name it trustpolicyforCWE.json.

      { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] }
    2. Use the following command to create the Role-for-MyRule role and attach the trust policy.

      Why am I making this change? Adding this trust policy to the role creates permissions for CloudWatch Events.

      aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforCWE.json
    3. Create the permissions policy JSON, as shown here for the pipeline named MyFirstPipeline. Name the permissions policy permissionspolicyforCWE.json.

      { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:80398EXAMPLE:MyFirstPipeline" ] } ] }
    4. Use the following command to attach the new CodePipeline-Permissions-Policy-for-CWE permissions policy to the Role-for-MyRule role you created.

      aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-CWE --policy-document file://permissionspolicyforCWE.json
  2. Call the put-rule command and include the --name and --event-pattern parameters.

    The following sample command uses --event-pattern to create a rule named MyS3SourceRule.

    aws events put-rule --name "MyS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"AWS API Call via CloudTrail\"],\"detail\":{\"eventSource\":[\"s3.amazonaws.com\"],\"eventName\":[\"PutObject\",\"CompleteMultiPartUpload\"],\"resources\":{\"ARN\":[\"arn:aws:s3:::myBucket/myFolder/file.zip\"]}}}
  3. To add AWS CodePipeline as a target, call the put-targets command and include the --rule and --targets parameters.

    The following command specifies that for the rule named MyS3SourceRule, the target Id is composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an example ARN for the pipeline. The pipeline starts when something changes in the repository.

    aws events put-targets --rule MyS3SourceRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline

To edit your pipeline's PollForSourceChanges parameter

  1. Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named MyFirstPipeline, run the following command:

    aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

    This command returns nothing, but the file you created should appear in the directory where you ran the command.

  2. Open the JSON file in any plain-text editor and edit the source stage by changing the PollForSourceChanges parameter for a bucket named storage-bucket to false, as shown in this example.

    Why am I making this change? Setting this parameter to false turns off periodic checks so you can use event-based change detection only.

    "configuration": { "S3Bucket": "storage-bucket", "PollForSourceChanges": "false", "S3ObjectKey": "index.zip" },
  3. If you are working with the pipeline structure retrieved using the get-pipeline command, you must remove the metadata lines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the "metadata": { } lines and the "created", "pipelineARN", and "updated" fields.

    For example, remove the following lines from the structure:

    "metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" }

    Save the file.

  4. To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:

    Important

    Be sure to include file:// before the file name. It is required in this command.

    aws codepipeline update-pipeline --cli-input-json file://pipeline.json

    This command returns the entire structure of the edited pipeline.

    Note

    The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.

Update GitHub Pipelines for Push Events (CLI)

Follow these steps to edit a pipeline that is using periodic checks to use a webhook instead. If you want to create a pipeline, see Create a Pipeline in AWS CodePipeline.

To build an event-driven pipeline, you edit the PollForSourceChanges parameter of your pipeline and then create the following resources manually:

  • GitHub webhook and authorization parameters

To create and register your webhook

  1. In a text editor, create and save a JSON file for the webhook you want to create. Use this sample file for a webhook named my-webhook:

    {"webhook": {"name": "my-webhook", "targetPipeline": "pipeline_name", "targetAction": "source_action_name", "filters": [ { "jsonPath": "$.ref", "matchEquals": "refs/heads/{Branch}" } ], "authentication": "GITHUB_HMAC", "authenticationConfiguration": {"SecretToken":"secret"} } }
  2. Call the put-webhook command and include the --cli-input and --region parameters.

    The following sample command creates a webhook with the webhook_json JSON file.

    aws codepipeline put-webhook --cli-input-json file://webhook_json.json --region "eu-central-1"
  3. In the output shown in this example, the URL and ARN are returned for a webhook named my-webhook.

    { "webhook": { "url": "https://webhooks.domain.com/trigger111111111EXAMPLE11111111111111111", "definition": { "authenticationConfiguration": { "SecretToken": "secret" }, "name": "my-webhook", "authentication": "GITHUB_HMAC", "targetPipeline": "pipeline_name", "targetAction": "Source", "filters": [ { "jsonPath": "$.ref", "matchEquals": "refs/heads/{Branch}" } ] }, "arn": "arn:aws:codepipeline:eu-central-1:ACCOUNT_ID:webhook:my-webhook" } }
  4. Call the register-webhook-with-third-party command and include the --webhook-name parameter.

    The following sample command registers a webhook named my-webhook.

    aws codepipeline register-webhook-with-third-party --webhook-name my-webhook

To edit your pipeline's PollForSourceChanges parameter

  1. Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named MyFirstPipeline, you would type the following command:

    aws codepipeline get-pipeline --name MyFirstPipeline >pipeline.json

    This command returns nothing, but the file you created should appear in the directory where you ran the command.

  2. Open the JSON file in any plain-text editor and edit the source stage by changing or adding the PollForSourceChanges parameter. In this example, for a repository named UserGitHubRepo, the parameter is set to false .

    Why am I making this change? Changing this parameter turns off periodic checks so you can use event-based change detection only.

    "configuration": { "Owner": "darlaker", "Repo": "UserGitHubRepo", "PollForSourceChanges": "false", "Branch": "master", "OAuthToken": "****" },
  3. If you are working with the pipeline structure retrieved using the get-pipeline command, you must edit the structure in the JSON file by removing the metadata lines from the file. Otherwise, the update-pipeline command cannot use it. Remove the "metadata" section from the pipeline structure in the JSON file, including the : { } and the "created", "pipelineARN", and "updated" fields.

    For example, remove the following lines from the structure:

    "metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" }

    Save the file.

  4. To apply your changes, run the update-pipeline command, specifying the pipeline JSON file, similar to the following:

    Important

    Be sure to include file:// before the file name. It is required in this command.

    aws codepipeline update-pipeline --cli-input-json file://pipeline.json

    This command returns the entire structure of the edited pipeline.

    Note

    The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.

Update Pipelines for Push Events (AWS CloudFormation Template)

You can use AWS CloudFormation to update your pipeline to use the recommended method to detect changes in your source.

Follow these steps to edit a pipeline that is using periodic checks to use . If you want to create a pipeline, see Continuous Delivery with AWS CodePipeline.

Update AWS CodeCommit Pipelines for Push Events (AWS CloudFormation Template)

To build an event-driven pipeline with AWS CodeCommit, you edit the PollForSourceChanges parameter of your pipeline and then add the following resources to your template:

  • An Amazon CloudWatch Events rule

  • An IAM role for your CloudWatch Events rule

If you use AWS CloudFormation to create and manage your pipelines, your template includes content like the following.

Note

The Configuration property in the source stage called PollForSourceChanges. If that property isn't included in your template, then PollForSourceChanges is set to true by default.

YAMLJSON
YAML
Resources: AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: codecommit-polling-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: CodeCommit OutputArtifacts: - Name: SourceOutput Configuration: BranchName: !Ref BranchName RepositoryName: !Ref RepositoryName PollForSourceChanges: true RunOrder: 1
JSON
"Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "CodeCommit" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "BranchName": { "Ref": "BranchName" }, "RepositoryName": { "Ref": "RepositoryName" }, "PollForSourceChanges": true }, "RunOrder": 1 } ] },

To update your pipeline AWS CloudFormation template and create CloudWatch Events rule

  1. In the template, under Resources, use the AWS::IAM::Role AWS CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:

    • The first policy allows the role to be assumed.

    • The second policy provides permissions to start the pipeline.

    Why am I making this change? Adding the AWS::IAM::Role resource enables AWS CloudFormation to create permissions for CloudWatch Events. This resource is added to your AWS CloudFormation stack.

    YAMLJSON
    YAML
    AmazonCloudWatchEventRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: cwe-pipeline-execution PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ]
    JSON
    "AmazonCloudWatchEventRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "cwe-pipeline-execution", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "codepipeline:StartPipelineExecution", "Resource": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ...
  2. In the template, under Resources, use the AWS::Events::Rule AWS CloudFormation resource to add a CloudWatch Events rule. This event pattern creates an event that monitors push changes to your repository When CloudWatch Events detects a repository state change, the rule invokes StartPipelineExecution on your target pipeline.

    Why am I making this change? Adding the AWS::Events::Rule resource enables AWS CloudFormation to create the event. This resource is added to your AWS CloudFormation stack.

    YAMLJSON
    YAML
    AmazonCloudWatchEventRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.codecommit detail-type: - 'CodeCommit Repository State Change' resources: - !Join [ '', [ 'arn:aws:codecommit:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref RepositoryName ] ] detail: event: - referenceCreated - referenceUpdated referenceType: - branch referenceName: - master Targets: - Arn: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn Id: codepipeline-AppPipeline
    JSON
    "AmazonCloudWatchEventRule": { "Type": "AWS::Events::Rule", "Properties": { "EventPattern": { "source": [ "aws.codecommit" ], "detail-type": [ "CodeCommit Repository State Change" ], "resources": [ { "Fn::Join": [ "", [ "arn:aws:codecommit:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "RepositoryName" } ] ] } ], "detail": { "event": [ "referenceCreated", "referenceUpdated" ], "referenceType": [ "branch" ], "referenceName": [ "master" ] } }, "Targets": [ { "Arn": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] }, "RoleArn": { "Fn::GetAtt": [ "AmazonCloudWatchEventRole", "Arn" ] }, "Id": "codepipeline-AppPipeline" } ] } },
  3. Save the updated template to your local computer, and then open the AWS CloudFormation console.

  4. Choose your stack, and then choose Create Change Set for Current Stack.

  5. Upload the template, and then view the changes listed in AWS CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.

  6. Choose Execute.

To edit your pipeline's PollForSourceChanges parameter

  • In the template, change PollForSourceChanges to false. If you did not include PollForSourceChanges in your pipeline definition, add it and set it to false.

    Why am I making this change? Changing this parameter to false turns off periodic checks so you can use event-based change detection only.

    YAMLJSON
    YAML
    Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: CodeCommit OutputArtifacts: - Name: SourceOutput Configuration: BranchName: !Ref BranchName RepositoryName: !Ref RepositoryName PollForSourceChanges: false RunOrder: 1
    JSON
    { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "CodeCommit" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "BranchName": { "Ref": "BranchName" }, "RepositoryName": { "Ref": "RepositoryName" }, "PollForSourceChanges": false }, "RunOrder": 1 } ] },

When you create these resources with AWS CloudFormation, your pipeline is triggered when files in your repository are created or updated. Here is the final template snippet:

YAMLJSON
YAML
Resources: AmazonCloudWatchEventRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: cwe-pipeline-execution PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] AmazonCloudWatchEventRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.codecommit detail-type: - 'CodeCommit Repository State Change' resources: - !Join [ '', [ 'arn:aws:codecommit:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref RepositoryName ] ] detail: event: - referenceCreated - referenceUpdated referenceType: - branch referenceName: - master Targets: - Arn: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn Id: codepipeline-AppPipeline AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: codecommit-events-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: CodeCommit OutputArtifacts: - Name: SourceOutput Configuration: BranchName: !Ref BranchName RepositoryName: !Ref RepositoryName PollForSourceChanges: false RunOrder: 1 ...
JSON
"Resources": { ... "AmazonCloudWatchEventRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "cwe-pipeline-execution", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "codepipeline:StartPipelineExecution", "Resource": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] } } ] } } ] } }, "AmazonCloudWatchEventRule": { "Type": "AWS::Events::Rule", "Properties": { "EventPattern": { "source": [ "aws.codecommit" ], "detail-type": [ "CodeCommit Repository State Change" ], "resources": [ { "Fn::Join": [ "", [ "arn:aws:codecommit:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "RepositoryName" } ] ] } ], "detail": { "event": [ "referenceCreated", "referenceUpdated" ], "referenceType": [ "branch" ], "referenceName": [ "master" ] } }, "Targets": [ { "Arn": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] }, "RoleArn": { "Fn::GetAtt": [ "AmazonCloudWatchEventRole", "Arn" ] }, "Id": "codepipeline-AppPipeline" } ] } }, "AppPipeline": { "Type": "AWS::CodePipeline::Pipeline", "Properties": { "Name": "codecommit-events-pipeline", "RoleArn": { "Fn::GetAtt": [ "CodePipelineServiceRole", "Arn" ] }, "Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "CodeCommit" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "BranchName": { "Ref": "BranchName" }, "RepositoryName": { "Ref": "RepositoryName" }, "PollForSourceChanges": false }, "RunOrder": 1 } ] }, ...

Update Amazon S3 Pipelines for Push Events (AWS CloudFormation Template)

Use these steps to edit your Amazon S3 pipeline from polling to event-based change detection.

To build an event-driven pipeline with Amazon S3, you edit the PollForSourceChanges parameter of your pipeline and then add the following resources to your template:

  • Amazon CloudWatch Events requires that all Amazon S3 events must be logged. You must create an AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the events that occur. For more information, see Logging Management and Data Events with AWS CloudTrail.

  • Amazon CloudWatch Events rule and IAM role to allow this event to start our pipeline.

If you use AWS CloudFormation to create and manage your pipelines, your template includes content like the following.

Note

The Configuration property in the source stage called PollForSourceChanges. If your template doesn't include that property, then PollForSourceChanges is set to true by default.

YAMLJSON
YAML
AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: S3 OutputArtifacts: - Name: SourceOutput Configuration: S3Bucket: !Ref SourceBucket S3ObjectKey: !Ref S3SourceObjectKey PollForSourceChanges: true RunOrder: 1 ...
JSON
"AppPipeline": { "Type": "AWS::CodePipeline::Pipeline", "Properties": { "RoleArn": { "Fn::GetAtt": ["CodePipelineServiceRole", "Arn"] }, "Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "S3" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "S3Bucket": { "Ref": "SourceBucket" }, "S3ObjectKey": { "Ref": "SourceObjectKey" }, "PollForSourceChanges": true }, "RunOrder": 1 } ] }, ...

To create a CloudWatch Events rule with Amazon S3 as the event source and AWS CodePipeline as the target and apply the permissions policy

  1. In the template, under Resources, use the AWS::IAM::Role AWS CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:

    • The first policy allows the role to be assumed.

    • The second policy provides permissions to start the pipeline.

    Why am I making this change? Adding AWS::IAM::Role resource enables AWS CloudFormation to create permissions for Amazon CloudWatch Events. This resource is added to your AWS CloudFormation stack.

    YAMLJSON
    YAML
    AmazonCloudWatchEventRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: cwe-pipeline-execution PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] ...
    JSON
    "AmazonCloudWatchEventRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "cwe-pipeline-execution", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "codepipeline:StartPipelineExecution", "Resource": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] ...
  2. Use the AWS::Events::Rule AWS CloudFormation resource to add a CloudWatch Events rule. This event pattern creates an event that monitors PutObject and CompleteMultipartUpload on your Amazon S3 source bucket. In addition, include a target of your pipeline. When PutObject or CompleteMultipartUpload occurs, this rule invokes StartPipelineExecution on your target pipeline.

    Why am I making this change? Adding the AWS::Events::Rule resource enables AWS CloudFormation to create the event. This resource is added to your AWS CloudFormation stack.

    YAMLJSON
    YAML
    AmazonCloudWatchEventRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.s3 detail-type: - 'AWS API Call via CloudTrail' detail: eventSource: - s3.amazonaws.com eventName: - PutObject - CompleteMultipartUpload resources: ARN: - !Join [ '', [ !GetAtt SourceBucket.Arn, '/', !Ref SourceObjectKey ] ] Targets: - Arn: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn Id: codepipeline-AppPipeline ...
    JSON
    "AmazonCloudWatchEventRule": { "Type": "AWS::Events::Rule", "Properties": { "EventPattern": { "source": [ "aws.s3" ], "detail-type": [ "AWS API Call via CloudTrail" ], "detail": { "eventSource": [ "s3.amazonaws.com" ], "eventName": [ "PutObject", "CompleteMultipartUpload" ], "resources": { "ARN": [ { "Fn::Join": [ "", [ { "Fn::GetAtt": [ "SourceBucket", "Arn" ] }, "/", { "Ref": "SourceObjectKey" } ] ] } ] } } }, "Targets": [ { "Arn": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] }, "RoleArn": { "Fn::GetAtt": [ "AmazonCloudWatchEventRole", "Arn" ] }, "Id": "codepipeline-AppPipeline" } ] } } }, ...
  3. Add this snippet to your first template to allow cross-stack functionality:

    YAMLJSON
    YAML
    Outputs: SourceBucketARN: Description: "S3 bucket ARN that Cloudtrail will use" Value: !GetAtt SourceBucket.Arn Export: Name: SourceBucketARN
    JSON
    "Outputs" : { "SourceBucketARN" : { "Description" : "S3 bucket ARN that Cloudtrail will use", "Value" : { "Fn::GetAtt": ["SourceBucket", "Arn"] }, "Export" : { "Name" : "SourceBucketARN" } } ...
  4. Save your updated template to your local computer, and open the AWS CloudFormation console.

  5. Choose your stack, and then choose Create Change Set for Current Stack.

  6. Upload your updated template, and then view the changes listed in AWS CloudFormation. These are the changes that will be made to the stack. You should see your new resources in the list.

  7. Choose Execute.

To edit your pipeline's PollForSourceChanges parameter

  • In the template, change PollForSourceChanges to false. If you did not include PollForSourceChanges in your pipeline definition, add it and set it to false.

    Why am I making this change? Changing PollForSourceChanges to false turns off periodic checks so you can use event-based change detection only.

    YAMLJSON
    YAML
    Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: S3 OutputArtifacts: - Name: SourceOutput Configuration: S3Bucket: !Ref SourceBucket S3ObjectKey: !Ref SourceObjectKey PollForSourceChanges: false RunOrder: 1
    JSON
    { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "S3" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "S3Bucket": { "Ref": "SourceBucket" }, "S3ObjectKey": { "Ref": "SourceObjectKey" }, "PollForSourceChanges": false }, "RunOrder": 1 }

To create a second template for your Amazon S3 pipeline's CloudTrail resources

  • In a separate template, under Resources, use the AWS::S3::Bucket, AWS::S3::BucketPolicy, and AWS::CloudTrail::Trail AWS CloudFormation resources to provide a simple bucket definition and trail for CloudTrail.

    Why am I making this change? Given the current limit of five trails per account, the CloudTrail trail must be created and managed separately. (See Limits in AWS CloudTrail.) However, you can include many Amazon S3 buckets on a single trail, so you can create the trail once and then add Amazon S3 buckets for other pipelines as necessary. Paste the following into your second sample template file.

    YAMLJSON
    YAML
    ################################################################################### # Prerequisites: # - S3 SoureBucket and SourceObjectKey must exist ################################################################################### Parameters: SourceObjectKey: Description: 'S3 source artifact' Type: String Default: SampleApp_Linux.zip Resources: AWSCloudTrailBucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref AWSCloudTrailBucket PolicyDocument: Version: 2012-10-17 Statement: - Sid: AWSCloudTrailAclCheck Effect: Allow Principal: Service: - cloudtrail.amazonaws.com Action: s3:GetBucketAcl Resource: !GetAtt AWSCloudTrailBucket.Arn - Sid: AWSCloudTrailWrite Effect: Allow Principal: Service: - cloudtrail.amazonaws.com Action: s3:PutObject Resource: !Join [ '', [ !GetAtt AWSCloudTrailBucket.Arn, '/AWSLogs/', !Ref 'AWS::AccountId', '/*' ] ] Condition: StringEquals: s3:x-amz-acl: bucket-owner-full-control AWSCloudTrailBucket: Type: AWS::S3::Bucket DeletionPolicy: Retain AwsCloudTrail: DependsOn: - AWSCloudTrailBucketPolicy Type: AWS::CloudTrail::Trail Properties: S3BucketName: !Ref AWSCloudTrailBucket EventSelectors: - DataResources: - Type: AWS::S3::Object Values: - !Join [ '', [ !ImportValue SourceBucketARN, '/', !Ref SourceObjectKey ] ] ReadWriteType: WriteOnly IncludeGlobalServiceEvents: true IsLogging: true IsMultiRegionTrail: true ...
    JSON
    { "Parameters": { "SourceObjectKey": { "Description": "S3 source artifact", "Type": "String", "Default": "SampleApp_Linux.zip" } }, "Resources": { "AWSCloudTrailBucket": { "Type": "AWS::S3::Bucket", "DeletionPolicy": "Retain" }, "AWSCloudTrailBucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": { "Ref": "AWSCloudTrailBucket" }, "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Sid": "AWSCloudTrailAclCheck", "Effect": "Allow", "Principal": { "Service": [ "cloudtrail.amazonaws.com" ] }, "Action": "s3:GetBucketAcl", "Resource": { "Fn::GetAtt": [ "AWSCloudTrailBucket", "Arn" ] } }, { "Sid": "AWSCloudTrailWrite", "Effect": "Allow", "Principal": { "Service": [ "cloudtrail.amazonaws.com" ] }, "Action": "s3:PutObject", "Resource": { "Fn::Join": [ "", [ { "Fn::GetAtt": [ "AWSCloudTrailBucket", "Arn" ] }, "/AWSLogs/", { "Ref": "AWS::AccountId" }, "/*" ] ] }, "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control" } } } ] } } }, "AwsCloudTrail": { "DependsOn": [ "AWSCloudTrailBucketPolicy" ], "Type": "AWS::CloudTrail::Trail", "Properties": { "S3BucketName": { "Ref": "AWSCloudTrailBucket" }, "EventSelectors": [ { "DataResources": [ { "Type": "AWS::S3::Object", "Values": [ { "Fn::Join": [ "", [ { "Fn::ImportValue": "SourceBucketARN" }, "/", { "Ref": "SourceObjectKey" } ] ] } ] } ], "ReadWriteType": "WriteOnly" } ], "IncludeGlobalServiceEvents": true, "IsLogging": true, "IsMultiRegionTrail": true } } } } ...

When you use AWS CloudFormation to create these resources, your pipeline is triggered when files in your repository are created or updated.

Note

Do not stop here. Although your pipeline is created, you must create a second AWS CloudFormation template for your Amazon S3 pipeline. If you do not create the second template, your pipeline does not have any change detection functionality.

YAMLJSON
YAML
Resources: SourceBucket: Type: AWS::S3::Bucket Properties: VersioningConfiguration: Status: Enabled CodePipelineArtifactStoreBucket: Type: AWS::S3::Bucket CodePipelineArtifactStoreBucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: !Ref CodePipelineArtifactStoreBucket PolicyDocument: Version: 2012-10-17 Statement: - Sid: DenyUnEncryptedObjectUploads Effect: Deny Principal: '*' Action: s3:PutObject Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ] Condition: StringNotEquals: s3:x-amz-server-side-encryption: aws:kms - Sid: DenyInsecureConnections Effect: Deny Principal: '*' Action: s3:* Resource: !Join [ '', [ !GetAtt CodePipelineArtifactStoreBucket.Arn, '/*' ] ] Condition: Bool: aws:SecureTransport: false CodePipelineServiceRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - codepipeline.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: AWS-CodePipeline-Service-3 PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: - codecommit:CancelUploadArchive - codecommit:GetBranch - codecommit:GetCommit - codecommit:GetUploadArchiveStatus - codecommit:UploadArchive Resource: '*' - Effect: Allow Action: - codedeploy:CreateDeployment - codedeploy:GetApplicationRevision - codedeploy:GetDeployment - codedeploy:GetDeploymentConfig - codedeploy:RegisterApplicationRevision Resource: '*' - Effect: Allow Action: - codebuild:BatchGetBuilds - codebuild:StartBuild Resource: '*' - Effect: Allow Action: - devicefarm:ListProjects - devicefarm:ListDevicePools - devicefarm:GetRun - devicefarm:GetUpload - devicefarm:CreateUpload - devicefarm:ScheduleRun Resource: '*' - Effect: Allow Action: - lambda:InvokeFunction - lambda:ListFunctions Resource: '*' - Effect: Allow Action: - iam:PassRole Resource: '*' - Effect: Allow Action: - elasticbeanstalk:* - ec2:* - elasticloadbalancing:* - autoscaling:* - cloudwatch:* - s3:* - sns:* - cloudformation:* - rds:* - sqs:* - ecs:* Resource: '*' AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: s3-events-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: AWS Version: 1 Provider: S3 OutputArtifacts: - Name: SourceOutput Configuration: S3Bucket: !Ref SourceBucket S3ObjectKey: !Ref SourceObjectKey PollForSourceChanges: false RunOrder: 1 - Name: Beta Actions: - Name: BetaAction InputArtifacts: - Name: SourceOutput ActionTypeId: Category: Deploy Owner: AWS Version: 1 Provider: CodeDeploy Configuration: ApplicationName: !Ref ApplicationName DeploymentGroupName: !Ref BetaFleet RunOrder: 1 ArtifactStore: Type: S3 Location: !Ref CodePipelineArtifactStoreBucket AmazonCloudWatchEventRole: Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Principal: Service: - events.amazonaws.com Action: sts:AssumeRole Path: / Policies: - PolicyName: cwe-pipeline-execution PolicyDocument: Version: 2012-10-17 Statement: - Effect: Allow Action: codepipeline:StartPipelineExecution Resource: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] AmazonCloudWatchEventRule: Type: AWS::Events::Rule Properties: EventPattern: source: - aws.s3 detail-type: - 'AWS API Call via CloudTrail' detail: eventSource: - s3.amazonaws.com eventName: - PutObject - CompleteMultipartUpload resources: ARN: - !Join [ '', [ !GetAtt SourceBucket.Arn, '/', !Ref SourceObjectKey ] ] Targets: - Arn: !Join [ '', [ 'arn:aws:codepipeline:', !Ref 'AWS::Region', ':', !Ref 'AWS::AccountId', ':', !Ref AppPipeline ] ] RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn Id: codepipeline-AppPipeline Outputs: SourceBucketARN: Description: "S3 bucket ARN that Cloudtrail will use" Value: !GetAtt SourceBucket.Arn Export: Name: SourceBucketARN
JSON
"Resources": { "SourceBucket": { "Type": "AWS::S3::Bucket", "Properties": { "VersioningConfiguration": { "Status": "Enabled" } } }, "CodePipelineArtifactStoreBucket": { "Type": "AWS::S3::Bucket" }, "CodePipelineArtifactStoreBucketPolicy": { "Type": "AWS::S3::BucketPolicy", "Properties": { "Bucket": { "Ref": "CodePipelineArtifactStoreBucket" }, "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Sid": "DenyUnEncryptedObjectUploads", "Effect": "Deny", "Principal": "*", "Action": "s3:PutObject", "Resource": { "Fn::Join": [ "", [ { "Fn::GetAtt": [ "CodePipelineArtifactStoreBucket", "Arn" ] }, "/*" ] ] }, "Condition": { "StringNotEquals": { "s3:x-amz-server-side-encryption": "aws:kms" } } }, { "Sid": "DenyInsecureConnections", "Effect": "Deny", "Principal": "*", "Action": "s3:*", "Resource": { "Fn::Join": [ "", [ { "Fn::GetAtt": [ "CodePipelineArtifactStoreBucket", "Arn" ] }, "/*" ] ] }, "Condition": { "Bool": { "aws:SecureTransport": false } } } ] } } }, "CodePipelineServiceRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "codepipeline.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "AWS-CodePipeline-Service-3", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codecommit:CancelUploadArchive", "codecommit:GetBranch", "codecommit:GetCommit", "codecommit:GetUploadArchiveStatus", "codecommit:UploadArchive" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "codedeploy:CreateDeployment", "codedeploy:GetApplicationRevision", "codedeploy:GetDeployment", "codedeploy:GetDeploymentConfig", "codedeploy:RegisterApplicationRevision" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "codebuild:BatchGetBuilds", "codebuild:StartBuild" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "devicefarm:ListProjects", "devicefarm:ListDevicePools", "devicefarm:GetRun", "devicefarm:GetUpload", "devicefarm:CreateUpload", "devicefarm:ScheduleRun" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "lambda:InvokeFunction", "lambda:ListFunctions" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "elasticbeanstalk:*", "ec2:*", "elasticloadbalancing:*", "autoscaling:*", "cloudwatch:*", "s3:*", "sns:*", "cloudformation:*", "rds:*", "sqs:*", "ecs:*" ], "Resource": "*" } ] } } ] } }, "AppPipeline": { "Type": "AWS::CodePipeline::Pipeline", "Properties": { "Name": "s3-events-pipeline", "RoleArn": { "Fn::GetAtt": [ "CodePipelineServiceRole", "Arn" ] }, "Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "AWS", "Version": 1, "Provider": "S3" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "S3Bucket": { "Ref": "SourceBucket" }, "S3ObjectKey": { "Ref": "SourceObjectKey" }, "PollForSourceChanges": false }, "RunOrder": 1 } ] }, { "Name": "Beta", "Actions": [ { "Name": "BetaAction", "InputArtifacts": [ { "Name": "SourceOutput" } ], "ActionTypeId": { "Category": "Deploy", "Owner": "AWS", "Version": 1, "Provider": "CodeDeploy" }, "Configuration": { "ApplicationName": { "Ref": "ApplicationName" }, "DeploymentGroupName": { "Ref": "BetaFleet" } }, "RunOrder": 1 } ] } ], "ArtifactStore": { "Type": "S3", "Location": { "Ref": "CodePipelineArtifactStoreBucket" } } } }, "AmazonCloudWatchEventRole": { "Type": "AWS::IAM::Role", "Properties": { "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "events.amazonaws.com" ] }, "Action": "sts:AssumeRole" } ] }, "Path": "/", "Policies": [ { "PolicyName": "cwe-pipeline-execution", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "codepipeline:StartPipelineExecution", "Resource": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] } } ] } } ] } }, "AmazonCloudWatchEventRule": { "Type": "AWS::Events::Rule", "Properties": { "EventPattern": { "source": [ "aws.s3" ], "detail-type": [ "AWS API Call via CloudTrail" ], "detail": { "eventSource": [ "s3.amazonaws.com" ], "eventName": [ "PutObject", "CompleteMultipartUpload" ], "resources": { "ARN": [ { "Fn::Join": [ "", [ { "Fn::GetAtt": [ "SourceBucket", "Arn" ] }, "/", { "Ref": "SourceObjectKey" } ] ] } ] } } }, "Targets": [ { "Arn": { "Fn::Join": [ "", [ "arn:aws:codepipeline:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":", { "Ref": "AppPipeline" } ] ] }, "RoleArn": { "Fn::GetAtt": [ "AmazonCloudWatchEventRole", "Arn" ] }, "Id": "codepipeline-AppPipeline" } ] } } }, "Outputs" : { "SourceBucketARN" : { "Description" : "S3 bucket ARN that Cloudtrail will use", "Value" : { "Fn::GetAtt": ["SourceBucket", "Arn"] }, "Export" : { "Name" : "SourceBucketARN" } } } } ...

Update GitHub Pipelines for Push Events (AWS CloudFormation Template)

Follow these steps to update your AWS CodeCommit pipeline from periodic checks (polling) to event-based change detection using Amazon CloudWatch Events.

To build an event-driven pipeline with AWS CodeCommit, you edit the PollForSourceChanges parameter of your pipeline and then add the following resources to your template:

  • A GitHub webhook

If you use AWS CloudFormation to create and manage your pipelines, your template has content like the following.

Note

Note the PollForSourceChanges configuration property in the source stage. If your template doesn't include that property, then PollForSourceChanges is set to true by default.

YAMLJSON
YAML
Resources: AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: github-polling-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: ThirdParty Version: 1 Provider: GitHub OutputArtifacts: - Name: SourceOutput Configuration: Owner: !Ref GitHubOwner Repo: !Ref RepositoryName Branch: !Ref BranchName OAuthToken: !Ref GitHubOAuthToken PollForSourceChanges: true RunOrder: 1 ...
JSON
"AppPipeline": { "Type": "AWS::CodePipeline::Pipeline", "Properties": { "Name": "github-polling-pipeline", "RoleArn": { "Fn::GetAtt": [ "CodePipelineServiceRole", "Arn" ] }, "Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "ThirdParty", "Version": 1, "Provider": "GitHub" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "Owner": { "Ref": "GitHubOwner" }, "Repo": { "Ref": "RepositoryName" }, "Branch": { "Ref": "BranchName" }, "OAuthToken": { "Ref": "GitHubOAuthToken" }, "PollForSourceChanges": true }, "RunOrder": 1 } ] }, ...

To add parameters and create a webhook in your template

  1. In the template, under Resources, add your parameters:

    YAMLJSON
    YAML
    Parameters: GitHubOwner: Type: String GitHubSecret: Type: String NoEcho: true GitHubOAuthToken: Type: String NoEcho: true ...
    JSON
    { "Parameters": { "BranchName": { "Description": "GitHub branch name", "Type": "String", "Default": "master" }, "GitHubOwner": { "Type": "String" }, "GitHubSecret": { "Type": "String", "NoEcho": true }, "GitHubOAuthToken": { "Type": "String", "NoEcho": true }, ...
  2. Use the AWS::CodePipeline::Webhook AWS CloudFormation resource to add a webhook.

    Note

    The TargetAction you specify must match the Name property of the source action defined in the pipeline.

    YAMLJSON
    YAML
    AppPipelineWebhook: Type: AWS::CodePipeline::Webhook Properties: Authentication: GITHUB_HMAC AuthenticationConfiguration: SecretToken: !Ref GitHubSecret Filters: - JsonPath: "$.ref" MatchEquals: refs/heads/{Branch} TargetPipeline: !Ref AppPipeline TargetAction: SourceAction Name: AppPipelineWebhook TargetPipelineVersion: !GetAtt AppPipeline.Version RegisterWithThirdParty: true ...
    JSON
    "AppPipelineWebhook": { "Type": "AWS::CodePipeline::Webhook", "Properties": { "Authentication": "GITHUB_HMAC", "AuthenticationConfiguration": { "SecretToken": { "Ref": "GitHubSecret" } }, "Filters": [ { "JsonPath": "$.ref", "MatchEquals": "refs/heads/{Branch}" } ], "TargetPipeline": { "Ref": "AppPipeline" }, "TargetAction": "SourceAction", "Name": "AppPipelineWebhook", "TargetPipelineVersion": { "Fn::GetAtt": [ "AppPipeline", "Version" ] }, "RegisterWithThirdParty": true } }, ...
  3. Save the updated template to your local computer, and then open the AWS CloudFormation console.

  4. Choose your stack, and then choose Create Change Set for Current Stack.

  5. Upload the template, and then view the changes listed in AWS CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.

  6. Choose Execute.

To edit your pipeline's PollForSourceChanges parameter

  • In the template, change PollForSourceChanges to false. If you did not include PollForSourceChanges in your pipeline definition, add it and set it to false.

    Why am I making this change? Changing this parameter to false turns off periodic checks so you can use event-based change detection only.

    YAMLJSON
    YAML
    Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: ThirdParty Version: 1 Provider: GitHub OutputArtifacts: - Name: SourceOutput Configuration: Owner: !Ref GitHubOwner Repo: !Ref RepositoryName Branch: !Ref BranchName OAuthToken: !Ref GitHubOAuthToken PollForSourceChanges: false RunOrder: 1
    JSON
    { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "ThirdParty", "Version": 1, "Provider": "GitHub" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "Owner": { "Ref": "GitHubOwner" }, "Repo": { "Ref": "RepositoryName" }, "Branch": { "Ref": "BranchName" }, "OAuthToken": { "Ref": "GitHubOAuthToken" }, "PollForSourceChanges": false }, "RunOrder": 1 }

When you create these resources with AWS CloudFormation, the webhook defined is created in the specified GitHub repository. Your pipeline is triggered on commit.

YAMLJSON
YAML
Parameters: GitHubOwner: Type: String GitHubSecret: Type: String NoEcho: true GitHubOAuthToken: Type: String NoEcho: true Resources: AppPipelineWebhook: Type: AWS::CodePipeline::Webhook Properties: Authentication: GITHUB_HMAC AuthenticationConfiguration: SecretToken: !Ref GitHubSecret Filters: - JsonPath: "$.ref" MatchEquals: refs/heads/{Branch} TargetPipeline: !Ref AppPipeline TargetAction: SourceAction Name: AppPipelineWebhook TargetPipelineVersion: !GetAtt AppPipeline.Version RegisterWithThirdParty: true AppPipeline: Type: AWS::CodePipeline::Pipeline Properties: Name: github-events-pipeline RoleArn: !GetAtt CodePipelineServiceRole.Arn Stages: - Name: Source Actions: - Name: SourceAction ActionTypeId: Category: Source Owner: ThirdParty Version: 1 Provider: GitHub OutputArtifacts: - Name: SourceOutput Configuration: Owner: !Ref GitHubOwner Repo: !Ref RepositoryName Branch: !Ref BranchName OAuthToken: !Ref GitHubOAuthToken PollForSourceChanges: false RunOrder: 1 ...
JSON
{ "Parameters": { "BranchName": { "Description": "GitHub branch name", "Type": "String", "Default": "master" }, "RepositoryName": { "Description": "GitHub repository name", "Type": "String", "Default": "test" }, "GitHubOwner": { "Type": "String" }, "GitHubSecret": { "Type": "String", "NoEcho": true }, "GitHubOAuthToken": { "Type": "String", "NoEcho": true }, "ApplicationName": { "Description": "CodeDeploy application name", "Type": "String", "Default": "DemoApplication" }, "BetaFleet": { "Description": "Fleet configured in CodeDeploy", "Type": "String", "Default": "DemoFleet" } }, "Resources": { ... }, "AppPipelineWebhook": { "Type": "AWS::CodePipeline::Webhook", "Properties": { "Authentication": "GITHUB_HMAC", "AuthenticationConfiguration": { "SecretToken": { "Ref": "GitHubSecret" } }, "Filters": [ { "JsonPath": "$.ref", "MatchEquals": "refs/heads/{Branch}" } ], "TargetPipeline": { "Ref": "AppPipeline" }, "TargetAction": "SourceAction", "Name": "AppPipelineWebhook", "TargetPipelineVersion": { "Fn::GetAtt": [ "AppPipeline", "Version" ] }, "RegisterWithThirdParty": true } }, "AppPipeline": { "Type": "AWS::CodePipeline::Pipeline", "Properties": { "Name": "github-events-pipeline", "RoleArn": { "Fn::GetAtt": [ "CodePipelineServiceRole", "Arn" ] }, "Stages": [ { "Name": "Source", "Actions": [ { "Name": "SourceAction", "ActionTypeId": { "Category": "Source", "Owner": "ThirdParty", "Version": 1, "Provider": "GitHub" }, "OutputArtifacts": [ { "Name": "SourceOutput" } ], "Configuration": { "Owner": { "Ref": "GitHubOwner" }, "Repo": { "Ref": "RepositoryName" }, "Branch": { "Ref": "BranchName" }, "OAuthToken": { "Ref": "GitHubOAuthToken" }, "PollForSourceChanges": false }, "RunOrder": 1 ...