Migrate polling pipelines to use event-based change detection
AWS CodePipeline supports full, end-to-end continuous delivery, which includes starting your pipeline whenever there is a code change. There are two supported ways to start your pipeline upon a code change: event-based change detection and polling. We recommend using event-based change detection for pipelines.
Use the procedures included here to migrate (update) your polling pipelines to the event-based change detection method for your pipeline.
The recommended event-based change detection method for pipelines is determined by the pipeline source, such as CodeCommit. In that case, for example, the polling pipeline would need to migrate to event-based change detection with EventBridge.
How to migrate polling pipelines
To migrate polling pipelines, determine your polling pipelines and then determine the recommended event-based change detection method:
-
Use the steps in Viewing polling pipelines in your account to determine your polling pipelines.
-
In the table, find your pipeline source type and then choose the procedure with the implementation you want to use to migrate your polling pipeline. Each section contains multiple methods for migration, such as using the CLI or AWS CloudFormation.
| How to migrate pipelines to the recommended change detection method | ||
|---|---|---|
| Pipeline source | Recommended event-based detection method | Migration procedures |
| AWS CodeCommit | EventBridge (recommended). | See Migrate polling pipelines with a CodeCommit source. |
| Amazon S3 | EventBridge and bucket enabled for event notifications (recommended). | See Migrate polling pipelines with an S3 source enabled for events. |
| Amazon S3 | EventBridge and an AWS CloudTrail trail. | See Migrate polling pipelines with an S3 source and CloudTrail trail. |
| GitHub version 1 | Connections (recommended) | See Migrate polling pipelines for a GitHub version 1 source action to connections. |
| GitHub version 1 | Webhooks | See Migrate polling pipelines for a GitHub version 1 source action to webhooks. |
Important
For applicable pipeline action configuration updates, such as pipelines with a GitHub
version 1 action, you must explicitly set the PollForSourceChanges
parameter to false within your Source action’s
configuration to stop a pipeline from polling. As a result, it is possible to
erroneously configure a pipeline with both event-based change detection and polling by, for example, configuring an EventBridge rule and
also omitting the PollForSourceChanges parameter. This results in duplicate
pipeline executions, and the pipeline is counted toward the limit on total number of
polling pipelines, which by default is much lower than event-based pipelines. For more
information, see Quotas in AWS CodePipeline.
Viewing polling pipelines in your account
As a first step, use one of the following scripts to determine which pipelines in your account are configured for polling. These are the pipelines to migrate to event-based change detection.
Viewing polling pipelines in your account (script)
Follow these steps to use a script to determine pipelines in your account that are using polling.
-
Open a terminal window, and then do one of the following:
-
Run the following command to create a new script named PollingPipelinesExtractor.sh.
vi PollingPipelinesExtractor.sh -
To use a python script, run the following command to create a new python script named PollingPipelinesExtractor.py.
vi PollingPipelinesExtractor.py
-
-
Copy and paste the following code into the PollingPipelinesExtractor script. Do one of the following:
-
Copy and paste the following code into the PollingPipelinesExtractor.sh script.
#!/bin/bash set +x POLLING_PIPELINES=() LAST_EXECUTED_DATES=() NEXT_TOKEN=null HAS_NEXT_TOKEN=true if [[ $# -eq 0 ]] ; then echo 'Please provide region name' exit 0 fi REGION=$1 while [ "$HAS_NEXT_TOKEN" != "false" ]; do if [ "$NEXT_TOKEN" != "null" ]; then LIST_PIPELINES_RESPONSE=$(aws codepipeline list-pipelines --region $REGION --next-token $NEXT_TOKEN) else LIST_PIPELINES_RESPONSE=$(aws codepipeline list-pipelines --region $REGION) fi LIST_PIPELINES=$(jq -r '.pipelines[].name' <<< "$LIST_PIPELINES_RESPONSE") NEXT_TOKEN=$(jq -r '.nextToken' <<< "$LIST_PIPELINES_RESPONSE") if [ "$NEXT_TOKEN" == "null" ]; then HAS_NEXT_TOKEN=false fi for pipline_name in $LIST_PIPELINES do PIPELINE=$(aws codepipeline get-pipeline --name $pipline_name --region $REGION) HAS_POLLABLE_ACTIONS=$(jq '.pipeline.stages[].actions[] | select(.actionTypeId.category == "Source") | select(.actionTypeId.owner == ("ThirdParty","AWS")) | select(.actionTypeId.provider == ("GitHub","S3","CodeCommit")) | select(.configuration.PollForSourceChanges == ("true",null))' <<< "$PIPELINE") if [ ! -z "$HAS_POLLABLE_ACTIONS" ]; then POLLING_PIPELINES+=("$pipline_name") PIPELINE_EXECUTIONS=$(aws codepipeline list-pipeline-executions --pipeline-name $pipline_name --region $REGION) LAST_EXECUTION=$(jq -r '.pipelineExecutionSummaries[0]' <<< "$PIPELINE_EXECUTIONS") if [ "$LAST_EXECUTION" != "null" ]; then LAST_EXECUTED_TIMESTAMP=$(jq -r '.startTime' <<< "$LAST_EXECUTION") LAST_EXECUTED_DATE="$(date -r ${LAST_EXECUTED_TIMESTAMP%.*})" else LAST_EXECUTED_DATE="Not executed in last year" fi LAST_EXECUTED_DATES+=("$LAST_EXECUTED_DATE") fi done done fileName=$REGION-$(date +%s) printf "| %-30s | %-30s |\n" "Polling Pipeline Name" "Last Executed Time" printf "| %-30s | %-30s |\n" "_____________________" "__________________" for i in "${!POLLING_PIPELINES[@]}"; do printf "| %-30s | %-30s |\n" "${POLLING_PIPELINES[i]}" "${LAST_EXECUTED_DATES[i]}" printf "${POLLING_PIPELINES[i]}," >> $fileName.csv done printf "\nSaving Polling Pipeline Names to file $fileName.csv." -
Copy and paste the following code into the PollingPipelinesExtractor.py script.
import boto3 import sys import time import math hasNextToken = True nextToken = "" pollablePipelines = [] lastExecutedTimes = [] if len(sys.argv) == 1: raise Exception("Please provide region name.") session = boto3.Session(profile_name='default', region_name=sys.argv[1]) codepipeline = session.client('codepipeline') def is_pollable_action(action): actionTypeId = action['actionTypeId'] configuration = action['configuration'] return actionTypeId['owner'] in {"AWS", "ThirdParty"} and actionTypeId['provider'] in {"GitHub", "CodeCommit", "S3"} and ('PollForSourceChanges' not in configuration or configuration['PollForSourceChanges'] == 'true') def has_pollable_actions(pipeline): hasPollableAction = False pipelineDefinition = codepipeline.get_pipeline(name=pipeline['name'])['pipeline'] for action in pipelineDefinition['stages'][0]['actions']: hasPollableAction = is_pollable_action(action) if hasPollableAction: break return hasPollableAction def get_last_executed_time(pipelineName): pipelineExecutions=codepipeline.list_pipeline_executions(pipelineName=pipelineName)['pipelineExecutionSummaries'] if pipelineExecutions: return pipelineExecutions[0]['startTime'].strftime("%A %m/%d/%Y, %H:%M:%S") else: return "Not executed in last year" while hasNextToken: if nextToken=="": list_pipelines_response = codepipeline.list_pipelines() else: list_pipelines_response = codepipeline.list_pipelines(nextToken=nextToken) if 'nextToken' in list_pipelines_response: nextToken = list_pipelines_response['nextToken'] else: hasNextToken= False for pipeline in list_pipelines_response['pipelines']: if has_pollable_actions(pipeline): pollablePipelines.append(pipeline['name']) lastExecutedTimes.append(get_last_executed_time(pipeline['name'])) fileName="{region}-{timeNow}.csv".format(region=sys.argv[1],timeNow=math.trunc(time.time())) file = open(fileName, 'w') print ("{:<30} {:<30} {:<30}".format('Polling Pipeline Name', '|','Last Executed Time')) print ("{:<30} {:<30} {:<30}".format('_____________________', '|','__________________')) for i in range(len(pollablePipelines)): print("{:<30} {:<30} {:<30}".format(pollablePipelines[i], '|', lastExecutedTimes[i])) file.write("{pipeline},".format(pipeline=pollablePipelines[i])) file.close() print("\nSaving Polling Pipeline Names to file {fileName}".format(fileName=fileName))
-
-
For each Region where you have pipelines, you must run the script for that Region. To run the script, do one of the following:
-
Run the following command to run the script named PollingPipelinesExtractor.sh. In this example, the Region is us-west-2.
./PollingPipelinesExtractor.sh us-west-2 -
For the python script, run the following command to run the python script namedPollingPipelinesExtractor.py. In this example, the Region is us-west-2.
python3 PollingPipelinesExtractor.py us-west-2
In the following sample output from the script, the Region us-west-2 returned a list of polling pipelines and shows the last execution time for each pipeline.
% ./pollingPipelineExtractor.sh us-west-2 | Polling Pipeline Name | Last Executed Time | | _____________________ | __________________ | | myCodeBuildPipeline | Wed Mar 8 09:35:49 PST 2023 | | myCodeCommitPipeline | Mon Apr 24 22:32:32 PDT 2023 | | TestPipeline | Not executed in last year | Saving list of polling pipeline names to us-west-2-1682496174.csv...%Analyze the script output and, for each pipeline in the list, update the polling source to the recommended event-based change detection method.
Note
Your polling pipelines are determined by the pipeline's action configuration for the
PollForSourceChangesparameter. If the pipeline source configuration has thePollForSourceChangesparameter ommitted, then CodePipeline defaults to polling your repository for source changes. This behavior is the same as ifPollForSourceChangesis included and set to true. For more information, see the configuration parameters for your pipeline's source action, such as the Amazon S3 source action configuration parameters in Amazon S3 source action reference.Note that this script also generates a .csv file containing the list of polling pipelines in your account and saves the .csv file to the current working folder.
-
Migrate polling pipelines with a CodeCommit source
You can migrate your polling pipeline to use EventBridge to detect changes in your CodeCommit source repository or your Amazon S3 source bucket.
CodeCommit -- For a pipeline with a CodeCommit source, modify the pipeline so that change detection is automated through EventBridge. Choose from the following methods to implement the migration:
Migrate polling pipelines (CodeCommit or Amazon S3 source) (console)
You can use the CodePipeline console to update your pipeline to use EventBridge to detect changes in your CodeCommit source repository or your Amazon S3 source bucket.
Note
When you use the console to edit a pipeline that has a CodeCommit source repository or an Amazon S3 source bucket, the rule and IAM role are created for you. If you use the AWS CLI to edit the pipeline, you must create the EventBridge rule and IAM role yourself. For more information, see CodeCommit source actions and EventBridge.
Use these steps to edit a pipeline that is using periodic checks. If you want to create a pipeline, see Create a pipeline, stages, and actions.
To edit the pipeline source stage
-
Sign in to the AWS Management Console and open the CodePipeline console at http://console.aws.amazon.com/codesuite/codepipeline/home
. The names of all pipelines associated with your AWS account are displayed.
-
In Name, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.
-
On the pipeline details page, choose Edit.
-
In Edit stage, choose the edit icon on the source action.
-
Expand Change Detection Options and choose Use CloudWatch Events to automatically start my pipeline when a change occurs (recommended).
A message appears showing the EventBridge rule to be created for this pipeline. Choose Update.
If you are updating a pipeline that has an Amazon S3 source, you see the following message. Choose Update.
-
When you have finished editing your pipeline, choose Save pipeline changes to return to the summary page.
A message displays the name of the EventBridge rule to be created for your pipeline. Choose Save and continue.
-
To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.
Migrate polling pipelines (CodeCommit source) (CLI)
Follow these steps to edit a pipeline that is using polling (periodic checks) to use an EventBridge rule to start the pipeline. If you want to create a pipeline, see Create a pipeline, stages, and actions.
To build an event-driven pipeline with CodeCommit, you edit the
PollForSourceChanges parameter of your pipeline and then create the
following resources:
-
EventBridge event
-
IAM role to allow this event to start your pipeline
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named
MyFirstPipeline, run the following command:aws codepipeline get-pipeline --nameMyFirstPipeline>pipeline.jsonThis command returns nothing, but the file you created should appear in the directory where you ran the command.
-
Open the JSON file in any plain-text editor and edit the source stage by changing the
PollForSourceChangesparameter tofalse, as shown in this example.Why am I making this change? Changing this parameter to
falseturns off periodic checks so you can use event-based change detection only."configuration": {"PollForSourceChanges": "false","BranchName": "main", "RepositoryName": "MyTestRepo" }, -
If you are working with the pipeline structure retrieved using the get-pipeline command, remove the
metadatalines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the"metadata": { }lines and the"created","pipelineARN", and"updated"fields.For example, remove the following lines from the structure:
"metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" },Save the file.
-
To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:
Important
Be sure to include
file://before the file name. It is required in this command.aws codepipeline update-pipeline --cli-input-json file://pipeline.jsonThis command returns the entire structure of the edited pipeline.
Note
The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the
start-pipeline-executioncommand to manually start your pipeline.
To create an EventBridge rule with CodeCommit as the event source and CodePipeline as the target
-
Add permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see Using resource-based policies for Amazon EventBridge.
-
Use the following sample to create the trust policy that allows EventBridge to assume the service role. Name the trust policy
trustpolicyforEB.json.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } -
Use the following command to create the
Role-for-MyRulerole and attach the trust policy.aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json -
Create the permissions policy JSON, as shown in this sample, for the pipeline named
MyFirstPipeline. Name the permissions policypermissionspolicyforEB.json.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:80398EXAMPLE:MyFirstPipeline" ] } ] } -
Use the following command to attach the
CodePipeline-Permissions-Policy-for-EBpermissions policy to theRole-for-MyRulerole.Why am I making this change? Adding this policy to the role creates permissions for EventBridge.
aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
-
-
Call the put-rule command and include the
--name,--event-pattern, and--role-arnparameters.Why am I making this change? This command enables AWS CloudFormation to create the event.
The following sample command creates a rule called
MyCodeCommitRepoRule.aws events put-rule --name "MyCodeCommitRepoRule" --event-pattern "{\"source\":[\"aws.codecommit\"],\"detail-type\":[\"CodeCommit Repository State Change\"],\"resources\":[\"repository-ARN\"],\"detail\":{\"referenceType\":[\"branch\"],\"referenceName\":[\"main\"]}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule" -
To add CodePipeline as a target, call the put-targets command and include the following parameters:
-
The
--ruleparameter is used with therule_nameyou created by using put-rule. -
The
--targetsparameter is used with the listIdof the target in the list of targets and theARNof the target pipeline.
The following sample command specifies that for the rule called
MyCodeCommitRepoRule, the targetIdis composed of the number one, indicating that in a list of targets for the rule, this is target 1. The sample command also specifies an exampleARNfor the pipeline. The pipeline starts when something changes in the repository.aws events put-targets --rule MyCodeCommitRepoRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline -
Migrate polling pipelines (CodeCommit source) (AWS CloudFormation template)
To build an event-driven pipeline with AWS CodeCommit, you edit the
PollForSourceChanges parameter of your pipeline and then add the
following resources to your template:
-
An EventBridge rule
-
An IAM role for your EventBridge rule
If you use AWS CloudFormation to create and manage your pipelines, your template includes content like the following.
Note
The Configuration property in the source stage called
PollForSourceChanges. If that property isn't included in your
template, then PollForSourceChanges is set to true by
default.
To update your pipeline AWS CloudFormation template and create EventBridge rule
-
In the template, under
Resources, use theAWS::IAM::RoleAWS CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:-
The first policy allows the role to be assumed.
-
The second policy provides permissions to start the pipeline.
Why am I making this change? Adding the
AWS::IAM::Roleresource enables AWS CloudFormation to create permissions for EventBridge. This resource is added to your AWS CloudFormation stack. -
-
In the template, under
Resources, use theAWS::Events::RuleAWS CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors push changes to your repository. When EventBridge detects a repository state change, the rule invokesStartPipelineExecutionon your target pipeline.Why am I making this change? Adding the
AWS::Events::Ruleresource enables AWS CloudFormation to create the event. This resource is added to your AWS CloudFormation stack. -
Save the updated template to your local computer, and then open the AWS CloudFormation console.
-
Choose your stack, and then choose Create Change Set for Current Stack.
-
Upload the template, and then view the changes listed in AWS CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.
-
Choose Execute.
To edit your pipeline's PollForSourceChanges parameter
Important
In many cases, the PollForSourceChanges parameter defaults to true when
you create a pipeline. When you add event-based change detection, you must add the
parameter to your output and set it to false to disable polling. Otherwise, your
pipeline starts twice for a single source change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
In the template, change
PollForSourceChangestofalse. If you did not includePollForSourceChangesin your pipeline definition, add it and set it tofalse.Why am I making this change? Changing this parameter to
falseturns off periodic checks so you can use event-based change detection only.
When you create these resources with AWS CloudFormation, your pipeline is triggered when files in your repository are created or updated. Here is the final template snippet:
Migrate polling pipelines with an S3 source enabled for events
For a pipeline with an Amazon S3 source, modify the pipeline so that change detection is automated through EventBridge and with a source bucket that is enabled for event notifications. This is the recommend method if you are using the CLI or AWS CloudFormation to migrate your pipeline.
Note
This includes using a bucket that is enabled for event notifications, where you do not need to create a separate CloudTrail trail. If you are using the console, then an event rule and CloudTrail trail are set up for you. For those steps, see Migrate polling pipelines with an S3 source and CloudTrail trail.
Migrate polling pipelines with an S3 source enabled for events (CLI)
Follow these steps to edit a pipeline that is using polling (periodic checks) to use an event in EventBridge instead. If you want to create a pipeline, see Create a pipeline, stages, and actions.
To build an event-driven pipeline with Amazon S3, you edit the
PollForSourceChanges parameter of your pipeline and then create the
following resources:
-
EventBridge event rule
-
IAM role to allow the EventBridge event to start your pipeline
To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy
-
Grant permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see Using resource-based policies for Amazon EventBridge.
-
Use the following sample to create the trust policy to allow EventBridge to assume the service role. Name it
trustpolicyforEB.json.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } -
Use the following command to create the
Role-for-MyRulerole and attach the trust policy.Why am I making this change? Adding this trust policy to the role creates permissions for EventBridge.
aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json -
Create the permissions policy JSON, as shown here for the pipeline named
MyFirstPipeline. Name the permissions policypermissionspolicyforEB.json.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:80398EXAMPLE:MyFirstPipeline" ] } ] } -
Use the following command to attach the new
CodePipeline-Permissions-Policy-for-EBpermissions policy to theRole-for-MyRulerole you created.aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
-
-
Call the put-rule command and include the
--name,--event-pattern, and--role-arnparameters.The following sample command creates a rule named
EnabledS3SourceRule.aws events put-rule --name "EnabledS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"Object Created\"],\"detail\":{\"bucket\":{\"name\":[\"amzn-s3-demo-source-bucket\"]}}}" --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule" -
To add CodePipeline as a target, call the put-targets command and include the
--ruleand--targetsparameters.The following command specifies that for the rule named
EnabledS3SourceRule, the targetIdis composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an exampleARNfor the pipeline. The pipeline starts when something changes in the repository.aws events put-targets --rule EnabledS3SourceRule --targets Id=codepipeline-AppPipeline,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named
MyFirstPipeline, run the following command:aws codepipeline get-pipeline --nameMyFirstPipeline>pipeline.jsonThis command returns nothing, but the file you created should appear in the directory where you ran the command.
-
Open the JSON file in any plain-text editor and edit the source stage by changing the
PollForSourceChangesparameter for a bucket namedamzn-s3-demo-source-buckettofalse, as shown in this example.Why am I making this change? Setting this parameter to
falseturns off periodic checks so you can use event-based change detection only."configuration": { "S3Bucket": "amzn-s3-demo-source-bucket","PollForSourceChanges": "false","S3ObjectKey": "index.zip" }, -
If you are working with the pipeline structure retrieved using the get-pipeline command, you must remove the
metadatalines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the"metadata": { }lines and the"created","pipelineARN", and"updated"fields.For example, remove the following lines from the structure:
"metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" },Save the file.
-
To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:
Important
Be sure to include
file://before the file name. It is required in this command.aws codepipeline update-pipeline --cli-input-json file://pipeline.jsonThis command returns the entire structure of the edited pipeline.
Note
The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.
Migrate polling pipelines with an S3 source enabled for events (AWS CloudFormation template)
This procedure is for a pipeline where the source bucket has events enabled.
Use these steps to edit your pipeline with an Amazon S3 source from polling to event-based change detection.
To build an event-driven pipeline with Amazon S3, you edit the
PollForSourceChanges parameter of your pipeline and then add the
following resources to your template:
-
EventBridge rule and IAM role to allow this event to start your pipeline.
If you use AWS CloudFormation to create and manage your pipelines, your template includes content like the following.
Note
The Configuration property in the source stage called
PollForSourceChanges. If your template doesn't include that
property, then PollForSourceChanges is set to true by
default.
To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy
-
In the template, under
Resources, use theAWS::IAM::RoleAWS CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:-
The first policy allows the role to be assumed.
-
The second policy provides permissions to start the pipeline.
Why am I making this change? Adding
AWS::IAM::Roleresource enables AWS CloudFormation to create permissions for EventBridge. This resource is added to your AWS CloudFormation stack. -
-
Use the
AWS::Events::RuleAWS CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitors creation or deletion of objects in your Amazon S3 source bucket. In addition, include a target of your pipeline. When an object is created, this rule invokesStartPipelineExecutionon your target pipeline.Why am I making this change? Adding the
AWS::Events::Ruleresource enables AWS CloudFormation to create the event. This resource is added to your AWS CloudFormation stack. -
Save your updated template to your local computer, and open the AWS CloudFormation console.
-
Choose your stack, and then choose Create Change Set for Current Stack.
-
Upload your updated template, and then view the changes listed in AWS CloudFormation. These are the changes that will be made to the stack. You should see your new resources in the list.
-
Choose Execute.
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
In the template, change
PollForSourceChangestofalse. If you did not includePollForSourceChangesin your pipeline definition, add it and set it tofalse.Why am I making this change? Changing
PollForSourceChangestofalseturns off periodic checks so you can use event-based change detection only.
When you use AWS CloudFormation to create these resources, your pipeline is triggered when files in your repository are created or updated.
Note
Do not stop here. Although your pipeline is created, you must create a second AWS CloudFormation template for your Amazon S3 pipeline. If you do not create the second template, your pipeline does not have any change detection functionality.
Migrate polling pipelines with an S3 source and CloudTrail trail
For a pipeline with an Amazon S3 source, modify the pipeline so that change detection is automated through EventBridge. Choose from the following methods to implement the migration:
Migrate polling pipelines with an S3 source and CloudTrail trail (CLI)
Follow these steps to edit a pipeline that is using polling (periodic checks) to use an event in EventBridge instead. If you want to create a pipeline, see Create a pipeline, stages, and actions.
To build an event-driven pipeline with Amazon S3, you edit the
PollForSourceChanges parameter of your pipeline and then create the
following resources:
-
AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the events.
-
EventBridge event
-
IAM role to allow the EventBridge event to start your pipeline
To create an AWS CloudTrail trail and enable logging
To use the AWS CLI to create a trail, call the create-trail command, specifying:
-
The trail name.
-
The bucket to which you have already applied the bucket policy for AWS CloudTrail.
For more information, see Creating a trail with the AWS command line interface.
-
Call the create-trail command and include the
--nameand--s3-bucket-nameparameters.Why am I making this change? This creates the CloudTrail trail required for your S3 source bucket.
The following command uses
--nameand--s3-bucket-nameto create a trail namedmy-trailand a bucket namedamzn-s3-demo-source-bucket.aws cloudtrail create-trail --name my-trail --s3-bucket-name amzn-s3-demo-source-bucket -
Call the start-logging command and include the
--nameparameter.Why am I making this change? This command starts the CloudTrail logging for your source bucket and sends events to EventBridge.
Example:
The following command uses
--nameto start logging on a trail namedmy-trail.aws cloudtrail start-logging --name my-trail -
Call the put-event-selectors command and include the
--trail-nameand--event-selectorsparameters. Use event selectors to specify that you want your trail to log data events for your source bucket and send the events to the EventBridge rule.Why am I making this change? This command filters events.
Example:
The following command uses
--trail-nameand--event-selectorsto specify data events for a source bucket and prefix namedamzn-s3-demo-source-bucket/myFolder.aws cloudtrail put-event-selectors --trail-namemy-trail--event-selectors '[{ "ReadWriteType": "WriteOnly", "IncludeManagementEvents":false, "DataResources": [{ "Type": "AWS::S3::Object", "Values": ["arn:aws:s3:::amzn-s3-demo-source-bucket/myFolder/file.zip"] }] }]'
To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy
-
Grant permissions for EventBridge to use CodePipeline to invoke the rule. For more information, see Using resource-based policies for Amazon EventBridge.
-
Use the following sample to create the trust policy to allow EventBridge to assume the service role. Name it
trustpolicyforEB.json.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "events.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } -
Use the following command to create the
Role-for-MyRulerole and attach the trust policy.Why am I making this change? Adding this trust policy to the role creates permissions for EventBridge.
aws iam create-role --role-name Role-for-MyRule --assume-role-policy-document file://trustpolicyforEB.json -
Create the permissions policy JSON, as shown here for the pipeline named
MyFirstPipeline. Name the permissions policypermissionspolicyforEB.json.{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "codepipeline:StartPipelineExecution" ], "Resource": [ "arn:aws:codepipeline:us-west-2:80398EXAMPLE:MyFirstPipeline" ] } ] } -
Use the following command to attach the new
CodePipeline-Permissions-Policy-for-EBpermissions policy to theRole-for-MyRulerole you created.aws iam put-role-policy --role-name Role-for-MyRule --policy-name CodePipeline-Permissions-Policy-For-EB --policy-document file://permissionspolicyforEB.json
-
-
Call the put-rule command and include the
--name,--event-pattern, and--role-arnparameters.The following sample command creates a rule named
MyS3SourceRule.aws events put-rule --name "MyS3SourceRule" --event-pattern "{\"source\":[\"aws.s3\"],\"detail-type\":[\"AWS API Call via CloudTrail\"],\"detail\":{\"eventSource\":[\"s3.amazonaws.com\"],\"eventName\":[\"CopyObject\",\"PutObject\",\"CompleteMultipartUpload\"],\"requestParameters\":{\"bucketName\":[\"amzn-s3-demo-source-bucket\"],\"key\":[\"my-key\"]}}} --role-arn "arn:aws:iam::ACCOUNT_ID:role/Role-for-MyRule" -
To add CodePipeline as a target, call the put-targets command and include the
--ruleand--targetsparameters.The following command specifies that for the rule named
MyS3SourceRule, the targetIdis composed of the number one, indicating that in a list of targets for the rule, this is target 1. The command also specifies an exampleARNfor the pipeline. The pipeline starts when something changes in the repository.aws events put-targets --rule MyS3SourceRule --targets Id=1,Arn=arn:aws:codepipeline:us-west-2:80398EXAMPLE:TestPipeline
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named
MyFirstPipeline, run the following command:aws codepipeline get-pipeline --nameMyFirstPipeline>pipeline.jsonThis command returns nothing, but the file you created should appear in the directory where you ran the command.
-
Open the JSON file in any plain-text editor and edit the source stage by changing the
PollForSourceChangesparameter for a bucket namedamzn-s3-demo-source-buckettofalse, as shown in this example.Why am I making this change? Setting this parameter to
falseturns off periodic checks so you can use event-based change detection only."configuration": { "S3Bucket": "amzn-s3-demo-source-bucket","PollForSourceChanges": "false","S3ObjectKey": "index.zip" }, -
If you are working with the pipeline structure retrieved using the get-pipeline command, you must remove the
metadatalines from the JSON file. Otherwise, the update-pipeline command cannot use it. Remove the"metadata": { }lines and the"created","pipelineARN", and"updated"fields.For example, remove the following lines from the structure:
"metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" },Save the file.
-
To apply your changes, run the update-pipeline command, specifying the pipeline JSON file:
Important
Be sure to include
file://before the file name. It is required in this command.aws codepipeline update-pipeline --cli-input-json file://pipeline.jsonThis command returns the entire structure of the edited pipeline.
Note
The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.
Migrate polling pipelines with an S3 source and CloudTrail trail (AWS CloudFormation template)
Use these steps to edit your pipeline with an Amazon S3 source from polling to event-based change detection.
To build an event-driven pipeline with Amazon S3, you edit the
PollForSourceChanges parameter of your pipeline and then add the
following resources to your template:
-
EventBridge requires that all Amazon S3 events must be logged. You must create an AWS CloudTrail trail, bucket, and bucket policy that Amazon S3 can use to log the events that occur. For more information, see Logging data events for trails and Logging management events for trails.
-
EventBridge rule and IAM role to allow this event to start our pipeline.
If you use AWS CloudFormation to create and manage your pipelines, your template includes content like the following.
Note
The Configuration property in the source stage called
PollForSourceChanges. If your template doesn't include that
property, then PollForSourceChanges is set to true by
default.
To create an EventBridge rule with Amazon S3 as the event source and CodePipeline as the target and apply the permissions policy
-
In the template, under
Resources, use theAWS::IAM::RoleAWS CloudFormation resource to configure the IAM role that allows your event to start your pipeline. This entry creates a role that uses two policies:-
The first policy allows the role to be assumed.
-
The second policy provides permissions to start the pipeline.
Why am I making this change? Adding
AWS::IAM::Roleresource enables AWS CloudFormation to create permissions for EventBridge. This resource is added to your AWS CloudFormation stack. -
-
Use the
AWS::Events::RuleAWS CloudFormation resource to add an EventBridge rule. This event pattern creates an event that monitorsCopyObject,PutObjectandCompleteMultipartUploadon your Amazon S3 source bucket. In addition, include a target of your pipeline. WhenCopyObject,PutObject, orCompleteMultipartUploadoccurs, this rule invokesStartPipelineExecutionon your target pipeline.Why am I making this change? Adding the
AWS::Events::Ruleresource enables AWS CloudFormation to create the event. This resource is added to your AWS CloudFormation stack. -
Add this snippet to your first template to allow cross-stack functionality:
-
Save your updated template to your local computer, and open the AWS CloudFormation console.
Choose your stack, and then choose Create Change Set for Current Stack.
Upload your updated template, and then view the changes listed in AWS CloudFormation. These are the changes that will be made to the stack. You should see your new resources in the list.
Choose Execute.
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
In the template, change
PollForSourceChangestofalse. If you did not includePollForSourceChangesin your pipeline definition, add it and set it tofalse.Why am I making this change? Changing
PollForSourceChangestofalseturns off periodic checks so you can use event-based change detection only.
To create a second template for your Amazon S3 pipeline's CloudTrail resources
-
In a separate template, under
Resources, use theAWS::S3::Bucket,AWS::S3::BucketPolicy, andAWS::CloudTrail::TrailAWS CloudFormation resources to provide a simple bucket definition and trail for CloudTrail.Why am I making this change? Given the current limit of five trails per account, the CloudTrail trail must be created and managed separately. (See Limits in AWS CloudTrail.) However, you can include many Amazon S3 buckets on a single trail, so you can create the trail once and then add Amazon S3 buckets for other pipelines as necessary. Paste the following into your second sample template file.
When you use AWS CloudFormation to create these resources, your pipeline is triggered when files in your repository are created or updated.
Note
Do not stop here. Although your pipeline is created, you must create a second AWS CloudFormation template for your Amazon S3 pipeline. If you do not create the second template, your pipeline does not have any change detection functionality.
Migrate polling pipelines for a GitHub version 1 source action to connections
You can migrate a GitHub version 1 source action to use connections for your external repository. This is the recommended change detection method for pipelines with a GitHub version 1 source action.
For a pipeline with a GitHub version 1 source action, we recommend modify the pipeline to use a GitHub version 2 action so that change detection is automated through AWS CodeConnections. For more information about working with connections, see GitHub connections.
Create a connection to GitHub (console)
You can use the console to create a connection to GitHub.
Step 1: Replace your version 1 GitHub action
Use the pipeline edit page to replace your version 1 GitHub action with a version 2 GitHub action.
To replace your version 1 GitHub action
-
Sign in to the CodePipeline console.
-
Choose your pipeline, and choose Edit. Choose Edit stage on your source stage. A message displays that recommends you update your action.
-
In Action provider, choose GitHub (Version 2).
-
Do one of the following:
-
Under Connection, if you have not already created a connection to your provider, choose Connect to GitHub. Proceed to Step 2: Create a connection to GitHub.
-
Under Connection, if you have already created a connection to your provider, choose the connection. Proceed to Step 3: Save the Source Action for Your Connection.
-
Step 2: Create a connection to GitHub
After you choose to create the connection, the Connect to GitHub page is shown.
To create a connection to GitHub
-
Under GitHub connection settings, your connection name is shown in Connection name.
Under GitHub Apps, choose an app installation or choose Install a new app to create one.
Note
You install one app for all of your connections to a particular provider. If you have already installed the GitHub app, choose it and skip this step.
-
If the authorization page for GitHub displays, log in with your credentials and then choose to continue.
-
On the app installation page, a message shows that the AWS CodeStar app is trying to connect to your GitHub account.
Note
You only install the app once for each GitHub account. If you previously installed the app, you can choose Configure to proceed to a modification page for your app installation, or you can use the back button to return to the console.
-
On the Install AWS CodeStar page, choose Install.
-
On the Connect to GitHub page, the connection ID for your new installation is displayed. Choose Connect.
Step 3: Save your GitHub source action
Complete your updates on the Edit action page to save your new source action.
To save your GitHub source action
-
In Repository, enter the name of your third-party repository. In Branch, enter the branch where you want your pipeline to detect source changes.
Note
In Repository, type
owner-name/repository-nameas shown in this example:my-account/my-repository -
In Output artifact format, choose the format for your artifacts.
-
To store output artifacts from the GitHub action using the default method, choose CodePipeline default. The action accesses the files from the GitHub repository and stores the artifacts in a ZIP file in the pipeline artifact store.
-
To store a JSON file that contains a URL reference to the repository so that downstream actions can perform Git commands directly, choose Full clone. This option can only be used by CodeBuild downstream actions.
If you choose this option, you will need to update the permissions for your CodeBuild project service role as shown in Add CodeBuild GitClone permissions for connections to Bitbucket, GitHub, GitHub Enterprise Server, or GitLab.com. For a tutorial that shows you how to use the Full clone option, see Tutorial: Use full clone with a GitHub pipeline source.
-
-
In Output artifacts, you can retain the name of the output artifact for this action, such as
SourceArtifact. Choose Done to close the Edit action page. -
Choose Done to close the stage editing page. Choose Save to close the pipeline editing page.
Create a connection to GitHub (CLI)
You can use the AWS Command Line Interface (AWS CLI) to create a connection to GitHub.
To do this, use the create-connection command.
Important
A connection created through the AWS CLI or AWS CloudFormation is in PENDING
status by default. After you create a connection with the CLI or AWS CloudFormation, use the
console to edit the connection to make its status AVAILABLE.
To create a connection to GitHub
-
Open a terminal (Linux, macOS, or Unix) or command prompt (Windows). Use the AWS CLI to run the create-connection command, specifying the
--provider-typeand--connection-namefor your connection. In this example, the third-party provider name isGitHuband the specified connection name isMyConnection.aws codeconnections create-connection --provider-type GitHub --connection-name MyConnectionIf successful, this command returns the connection ARN information similar to the following.
{ "ConnectionArn": "arn:aws:codeconnections:us-west-2:account_id:connection/aEXAMPLE-8aad-4d5d-8878-dfcab0bc441f" } -
Use the console to complete the connection.
Migrate polling pipelines for a GitHub version 1 source action to webhooks
You can migrate your pipeline to use webhooks to detect changes in your GitHub source repository. This migration to webhooks is for the GitHub version 1 action only.
Important
When creating CodePipeline webhooks, do not use your own credentials or reuse the same secret token across multiple webhooks. For optimal security, generate a unique secret token for each webhook you create. The secret token is an arbitrary string that you provide, which GitHub uses to compute and sign the webhook payloads sent to CodePipeline, for protecting the integrity and authenticity of the webhook payloads. Using your own credentials or reusing the same token across multiple webhooks can lead to security vulnerabilities.
Migrate polling pipelines to webhooks (GitHub version 1 source actions) (console)
For the GitHub version 1 source action, you can use the CodePipeline console to update your pipeline to use webhooks to detect changes in your GitHub source repository.
Follow these steps to edit a pipeline that is using polling (periodic checks) to use EventBridge instead. If you want to create a pipeline, see Create a pipeline, stages, and actions.
When you use the console, the PollForSourceChanges parameter for your
pipelined is changed for you. The GitHub webhook is created and registered for
you.
To edit the pipeline source stage
-
Sign in to the AWS Management Console and open the CodePipeline console at http://console.aws.amazon.com/codesuite/codepipeline/home
. The names of all pipelines associated with your AWS account are displayed.
-
In Name, choose the name of the pipeline you want to edit. This opens a detailed view of the pipeline, including the state of each of the actions in each stage of the pipeline.
-
On the pipeline details page, choose Edit.
-
In Edit stage, choose the edit icon on the source action.
-
Expand Change detection options and choose Use Amazon CloudWatch Events to automatically start my pipeline when a change occurs (recommended).
A message is displayed to advise that CodePipeline creates a webhook in GitHub to detect source changes: AWS CodePipeline will create a webhook for you. You can opt-out in the options below. Choose Update. In addition to the webhook, CodePipeline creates the following:
-
A secret, randomly generated and used to authorize the connection to GitHub.
-
The webhook URL, generated using the public endpoint for the Region.
CodePipeline registers the webhook with GitHub. This subscribes the URL to receive repository events.
-
-
When you have finished editing your pipeline, choose Save pipeline changes to return to the summary page.
A message displays the name of the webhook to be created for your pipeline. Choose Save and continue.
-
To test your action, release a change by using the AWS CLI to commit a change to the source specified in the source stage of the pipeline.
Migrate polling pipelines to webhooks (GitHub version 1 source actions) (CLI)
Follow these steps to edit a pipeline that is using periodic checks to use a webhook instead. If you want to create a pipeline, see Create a pipeline, stages, and actions.
To build an event-driven pipeline, you edit the PollForSourceChanges
parameter of your pipeline and then create the following resources manually:
-
GitHub webhook and authorization parameters
To create and register your webhook
Note
When you use the CLI or AWS CloudFormation to create a pipeline and add a webhook, you must disable
periodic checks. To disable periodic checks, you must explicitly add the
PollForSourceChanges parameter and set it to false, as detailed in the
final procedure below. Otherwise, the default for a CLI or AWS CloudFormation pipeline is that
PollForSourceChanges defaults to true and does not display in the
pipeline structure output. For more information about PollForSourceChanges defaults, see
Valid settings for the
PollForSourceChanges parameter.
-
In a text editor, create and save a JSON file for the webhook you want to create. Use this sample file for a webhook named
my-webhook:{ "webhook": { "name": "my-webhook", "targetPipeline": "pipeline_name", "targetAction": "source_action_name", "filters": [{ "jsonPath": "$.ref", "matchEquals": "refs/heads/{Branch}" }], "authentication": "GITHUB_HMAC", "authenticationConfiguration": { "SecretToken": "secret" } } } -
Call the put-webhook command and include the
--cli-inputand--regionparameters.The following sample command creates a webhook with the
webhook_jsonJSON file.aws codepipeline put-webhook --cli-input-json file://webhook_json.json --region "eu-central-1" -
In the output shown in this example, the URL and ARN are returned for a webhook named
my-webhook.{ "webhook": { "url": "https://webhooks.domain.com/trigger111111111EXAMPLE11111111111111111", "definition": { "authenticationConfiguration": { "SecretToken": "secret" }, "name": "my-webhook", "authentication": "GITHUB_HMAC", "targetPipeline": "pipeline_name", "targetAction": "Source", "filters": [ { "jsonPath": "$.ref", "matchEquals": "refs/heads/{Branch}" } ] }, "arn": "arn:aws:codepipeline:eu-central-1:ACCOUNT_ID:webhook:my-webhook" }, "tags": [{ "key": "Project", "value": "ProjectA" }] }This example adds tagging to the webhook by including the
Projecttag key andProjectAvalue on the webhook. For more information about tagging resources in CodePipeline, see Tagging resources. -
Call the register-webhook-with-third-party command and include the
--webhook-nameparameter.The following sample command registers a webhook named
my-webhook.aws codepipeline register-webhook-with-third-party --webhook-name my-webhook
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
Run the get-pipeline command to copy the pipeline structure into a JSON file. For example, for a pipeline named
MyFirstPipeline, you would type the following command:aws codepipeline get-pipeline --nameMyFirstPipeline>pipeline.jsonThis command returns nothing, but the file you created should appear in the directory where you ran the command.
-
Open the JSON file in any plain-text editor and edit the source stage by changing or adding the
PollForSourceChangesparameter. In this example, for a repository namedUserGitHubRepo, the parameter is set tofalse.Why am I making this change? Changing this parameter turns off periodic checks so you can use event-based change detection only.
"configuration": { "Owner": "name", "Repo": "UserGitHubRepo","PollForSourceChanges": "false","Branch": "main", "OAuthToken": "****" }, -
If you are working with the pipeline structure retrieved using the get-pipeline command, you must edit the structure in the JSON file by removing the
metadatalines from the file. Otherwise, the update-pipeline command cannot use it. Remove the"metadata"section from the pipeline structure in the JSON file, including the :{ }and the"created","pipelineARN", and"updated"fields.For example, remove the following lines from the structure:
"metadata": { "pipelineArn": "arn:aws:codepipeline:region:account-ID:pipeline-name", "created": "date", "updated": "date" },Save the file.
-
To apply your changes, run the update-pipeline command, specifying the pipeline JSON file, similar to the following:
Important
Be sure to include
file://before the file name. It is required in this command.aws codepipeline update-pipeline --cli-input-json file://pipeline.jsonThis command returns the entire structure of the edited pipeline.
Note
The update-pipeline command stops the pipeline. If a revision is being run through the pipeline when you run the update-pipeline command, that run is stopped. You must manually start the pipeline to run that revision through the updated pipeline. Use the start-pipeline-execution command to manually start your pipeline.
Update pipelines for push events (GitHub version 1 source actions) (AWS CloudFormation template)
Follow these steps to update your pipeline (with a GitHub source) from periodic checks (polling) to event-based change detection using webhooks.
To build an event-driven pipeline with AWS CodeCommit, you edit the
PollForSourceChanges parameter of your pipeline and then add a
GitHub webhook resource to your template.
If you use AWS CloudFormation to create and manage your pipelines, your template has content like the following.
Note
Note the PollForSourceChanges configuration property in the
source stage. If your template doesn't include that property, then
PollForSourceChanges is set to true by
default.
To add parameters and create a webhook in your template
We strongly recommend that you use AWS Secrets Manager to store your credentials. If you use Secrets Manager, you must have already configured and stored your secret parameters in Secrets Manager. This example uses dynamic references to Secrets Manager for the GitHub credentials for your webhook. For more information, see Using Dynamic References to Specify Template Values.
Important
When passing secret parameters, do not enter the value directly into the template. The value is rendered as plaintext and is therefore readable. For security reasons, do not use plaintext in your AWS CloudFormation template to store your credentials.
When you use the CLI or AWS CloudFormation to create a pipeline and add a webhook, you must disable periodic checks.
Note
To disable periodic checks, you must explicitly add the
PollForSourceChanges parameter and set it to false, as detailed in the
final procedure below. Otherwise, the default for a CLI or AWS CloudFormation pipeline is that
PollForSourceChanges defaults to true and does not display in the
pipeline structure output. For more information about PollForSourceChanges defaults, see
Valid settings for the
PollForSourceChanges parameter.
-
In the template, under
Resources, add your parameters: -
Use the
AWS::CodePipeline::WebhookAWS CloudFormation resource to add a webhook.Note
The
TargetActionyou specify must match theNameproperty of the source action defined in the pipeline.If
RegisterWithThirdPartyis set totrue, make sure the user associated to theOAuthTokencan set the required scopes in GitHub. The token and webhook require the following GitHub scopes:-
repo- used for full control to read and pull artifacts from public and private repositories into a pipeline. -
admin:repo_hook- used for full control of repository hooks.
Otherwise, GitHub returns a 404. For more information about the 404 returned, see https://help.github.com/articles/about-webhooks
. -
-
Save the updated template to your local computer, and then open the AWS CloudFormation console.
-
Choose your stack, and then choose Create Change Set for Current Stack.
-
Upload the template, and then view the changes listed in AWS CloudFormation. These are the changes to be made to the stack. You should see your new resources in the list.
-
Choose Execute.
To edit your pipeline's PollForSourceChanges parameter
Important
When you create a pipeline with this method, the PollForSourceChanges
parameter defaults to true if it is not explicitly set to false. When you add
event-based change detection, you must add the parameter to your output and set it to
false to disable polling. Otherwise, your pipeline starts twice for a single source
change. For details, see Valid settings for the
PollForSourceChanges parameter.
-
In the template, change
PollForSourceChangestofalse. If you did not includePollForSourceChangesin your pipeline definition, add it and set it to false.Why am I making this change? Changing this parameter to
falseturns off periodic checks so you can use event-based change detection only.
When you create these resources with AWS CloudFormation, the webhook defined is created in the specified GitHub repository. Your pipeline is triggered on commit.
