Docker samples for CodeBuild - AWS CodeBuild

Docker samples for CodeBuild

Docker in custom image sample for CodeBuild

This sample builds and runs a Docker image by using AWS CodeBuild and a custom Docker build image (docker:dind in Docker Hub).

To learn how to build a Docker image by using a build image provided by CodeBuild with Docker support instead, see our Publish Docker image to an Amazon ECR image repository sample.

Important

Running this sample might result in charges to your AWS account. These include possible charges for CodeBuild and for AWS resources and actions related to Amazon S3, AWS KMS, and CloudWatch Logs. For more information, see CodeBuild pricing, Amazon S3 pricing, AWS Key Management Service pricing, and Amazon CloudWatch pricing.

Running the sample

To run this sample
  1. Create the files as described in the "Directory structure" and "Files" sections of this topic, and then upload them to an S3 input bucket or an AWS CodeCommit, GitHub, or Bitbucket repository.

    Important

    Do not upload (root directory name), just the files inside of (root directory name).

    If you are using an S3 input bucket, be sure to create a ZIP file that contains the files, and then upload it to the input bucket. Do not add (root directory name) to the ZIP file, just the files inside of (root directory name).

  2. Create a build project, run the build, and view related build information.

    If you use the AWS CLI to create the build project, the JSON-formatted input to the create-project command might look similar to this. (Replace the placeholders with your own values.)

    { "name": "sample-docker-custom-image-project", "source": { "type": "S3", "location": "codebuild-region-ID-account-ID-input-bucket/DockerCustomImageSample.zip" }, "artifacts": { "type": "NO_ARTIFACTS" }, "environment": { "type": "LINUX_CONTAINER", "image": "docker:dind", "computeType": "BUILD_GENERAL1_SMALL", "privilegedMode": false }, "serviceRole": "arn:aws:iam::account-ID:role/role-name", "encryptionKey": "arn:aws:kms:region-ID:account-ID:key/key-ID" }
    Note

    By default, Docker daemon is enabled for non-VPC builds. If you would like to use Docker containers for VPC builds, see Runtime Privilege and Linux Capabilities on the Docker Docs website and enable privileged mode. Also, Windows does not support privileged mode.

  3. To see the build results, look in the build's log for the string Hello, World!. For more information, see View build details.

Directory structure

This sample assumes this directory structure.

(root directory name) ├── buildspec.yml └── Dockerfile

Files

The base image of the operating system used in this sample is Ubuntu. The sample uses these files.

buildspec.yml (in (root directory name))

version: 0.2 phases: pre_build: commands: - docker build -t helloworld . build: commands: - docker images - docker run helloworld echo "Hello, World!"

Dockerfile (in (root directory name))

FROM maven:3.3.9-jdk-8 RUN echo "Hello World"

Related resources

Publish Docker image to an Amazon Elastic Container Registry image repository sample for CodeBuild

This sample produces as build output a Docker image and then pushes the Docker image to an Amazon Elastic Container Registry (Amazon ECR) image repository. You can adapt this sample to push the Docker image to Docker Hub. For more information, see Adapting the sample to push the image to Docker Hub.

To learn how to build a Docker image by using a custom Docker build image (docker:dind in Docker Hub), see our Docker in custom image sample.

This sample was tested referencing golang:1.12.

This sample uses the new multi-stage Docker builds feature, which produces a Docker image as build output. It then pushes the Docker image to an Amazon ECR image repository. Multi-stage Docker image builds help to reduce the size of the final Docker image. For more information, see Use multi-stage builds with Docker.

Important

Running this sample might result in charges to your AWS account. These include possible charges for AWS CodeBuild and for AWS resources and actions related to Amazon S3, AWS KMS, CloudWatch Logs, and Amazon ECR. For more information, see CodeBuild pricing, Amazon S3 pricing, AWS Key Management Service pricing, Amazon CloudWatch pricing, and Amazon Elastic Container Registry pricing.

Running the sample

To run this sample
  1. If you already have an image repository in Amazon ECR you want to use, skip to step 3. Otherwise, if you are using an user instead of an AWS root account or an administrator user to work with Amazon ECR, add this statement (between ### BEGIN ADDING STATEMENT HERE ### and ### END ADDING STATEMENT HERE ###) to the user (or IAM group the user is associated with). Using an AWS root account is not recommended.This statement allows the creation of Amazon ECR repositories for storing Docker images. Ellipses (...) are used for brevity and to help you locate where to add the statement. Do not remove any statements, and do not type these ellipses into the policy. For more information, see Working with inline policies using the AWS Management Console in the user Guide.

    { "Statement": [ ### BEGIN ADDING STATEMENT HERE ### { "Action": [ "ecr:CreateRepository" ], "Resource": "*", "Effect": "Allow" }, ### END ADDING STATEMENT HERE ### ... ], "Version": "2012-10-17" }
    Note

    The IAM entity that modifies this policy must have permission in IAM to modify policies.

  2. Create an image repository in Amazon ECR. Be sure to create the repository in the same AWS Region where you create your build environment and run your build. For more information, see Creating a repository in the Amazon ECR User Guide. This repository's name must match the repository name you specify later in this procedure, represented by the IMAGE_REPO_NAME environment variable. Ensure that the Amazon ECR repository policy grants image push access for your CodeBuild service IAM role.

  3. Add this statement (between ### BEGIN ADDING STATEMENT HERE ### and ### END ADDING STATEMENT HERE ###) to the policy you attached to your AWS CodeBuild service role. This statement allows CodeBuild to upload Docker images to Amazon ECR repositories. Ellipses (...) are used for brevity and to help you locate where to add the statement. Do not remove any statements, and do not type these ellipses into the policy.

    { "Statement": [ ### BEGIN ADDING STATEMENT HERE ### { "Action": [ "ecr:BatchCheckLayerAvailability", "ecr:CompleteLayerUpload", "ecr:GetAuthorizationToken", "ecr:InitiateLayerUpload", "ecr:PutImage", "ecr:UploadLayerPart" ], "Resource": "*", "Effect": "Allow" }, ### END ADDING STATEMENT HERE ### ... ], "Version": "2012-10-17" }
    Note

    The IAM entity that modifies this policy must have permission in IAM to modify policies.

  4. Create the files as described in the "Directory structure" and "Files" sections of this topic, and then upload them to an S3 input bucket or an AWS CodeCommit, GitHub, or Bitbucket repository. For more information, see Image definitions file reference in the AWS CodePipeline User Guide.

    Important

    Do not upload (root directory name), just the files inside of (root directory name).

    If you are using an S3 input bucket, be sure to create a ZIP file that contains the files, and then upload it to the input bucket. Do not add (root directory name) to the ZIP file, just the files inside of (root directory name).

  5. Create a build project, run the build, and view build information.

    If you use the console to create your project:

    1. For Operating system, choose Ubuntu.

    2. For Runtime, choose Standard.

    3. For Image, choose aws/codebuild/standard:5.0.

    4. Add the following environment variables:

      • AWS_DEFAULT_REGION with a value of region-ID

      • AWS_ACCOUNT_ID with a value of account-ID

      • IMAGE_TAG with a value of Latest

      • IMAGE_REPO_NAME with a value of Amazon-ECR-repo-name

    If you use the AWS CLI to create the build project, the JSON-formatted input to the create-project command might look similar to this. (Replace the placeholders with your own values.)

    { "name": "sample-docker-project", "source": { "type": "S3", "location": "codebuild-region-ID-account-ID-input-bucket/DockerSample.zip" }, "artifacts": { "type": "NO_ARTIFACTS" }, "environment": { "type": "LINUX_CONTAINER", "image": "aws/codebuild/standard:5.0", "computeType": "BUILD_GENERAL1_SMALL", "environmentVariables": [ { "name": "AWS_DEFAULT_REGION", "value": "region-ID" }, { "name": "AWS_ACCOUNT_ID", "value": "account-ID" }, { "name": "IMAGE_REPO_NAME", "value": "Amazon-ECR-repo-name" }, { "name": "IMAGE_TAG", "value": "latest" } ], }, "serviceRole": "arn:aws:iam::account-ID:role/role-name", "encryptionKey": "arn:aws:kms:region-ID:account-ID:key/key-ID" }
  6. Confirm that CodeBuild successfully pushed the Docker image to the repository:

    1. Open the Amazon ECR console at https://console.aws.amazon.com/ecr/.

    2. Choose the repository name. The image should be listed in the Image tag column.

Directory structure

This sample assumes this directory structure.

(root directory name) ├── buildspec.yml └── Dockerfile

Files

This sample uses these files.

buildspec.yml (in (root directory name))

version: 0.2 phases: pre_build: commands: - echo Logging in to Amazon ECR... - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG . - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker image... - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG

Dockerfile (in (root directory name))

FROM golang:1.12-alpine AS build #Install git RUN apk add --no-cache git #Get the hello world package from a GitHub repository RUN go get github.com/golang/example/hello WORKDIR /go/src/github.com/golang/example/hello # Build the project and send the output to /bin/HelloWorld RUN go build -o /bin/HelloWorld FROM golang:1.12-alpine #Copy the build's output binary from the previous build container COPY --from=build /bin/HelloWorld /bin/HelloWorld ENTRYPOINT ["/bin/HelloWorld"]
Note

CodeBuild overrides the ENTRYPOINT for custom Docker images.

Adapting the sample to push the image to Docker Hub

To push the Docker image to Docker Hub instead of Amazon ECR, edit this sample's code.

Note

If you are using a version of Docker earlier than 17.06, remove the --no-include-email option.

  1. Replace these Amazon ECR-specific lines of code in the buildspec.yml file:

    ... pre_build: commands: - echo Logging in to Amazon ECR... - aws ecr get-login-password --region $AWS_DEFAULT_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG . - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker image... - docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_DEFAULT_REGION.amazonaws.com/$IMAGE_REPO_NAME:$IMAGE_TAG ...

    With these Docker Hub-specific lines of code:

    ... pre_build: commands: - echo Logging in to Docker Hub... # Type the command to log in to your Docker Hub account here. build: commands: - echo Build started on `date` - echo Building the Docker image... - docker build -t $IMAGE_REPO_NAME:$IMAGE_TAG . - docker tag $IMAGE_REPO_NAME:$IMAGE_TAG $IMAGE_REPO_NAME:$IMAGE_TAG post_build: commands: - echo Build completed on `date` - echo Pushing the Docker image... - docker push $IMAGE_REPO_NAME:$IMAGE_TAG ...
  2. Upload the edited code to an S3 input bucket or an AWS CodeCommit, GitHub, or Bitbucket repository.

    Important

    Do not upload (root directory name), just the files inside of (root directory name).

    If you are using an S3 input bucket, be sure to create a ZIP file that contains the files, and then upload it to the input bucket. Do not add (root directory name) to the ZIP file, just the files inside of (root directory name).

  3. Replace these lines of code from the JSON-formatted input to the create-project command:

    ... "environmentVariables": [ { "name": "AWS_DEFAULT_REGION", "value": "region-ID" }, { "name": "AWS_ACCOUNT_ID", "value": "account-ID" }, { "name": "IMAGE_REPO_NAME", "value": "Amazon-ECR-repo-name" }, { "name": "IMAGE_TAG", "value": "latest" } ] ...

    With these lines of code:

    ... "environmentVariables": [ { "name": "IMAGE_REPO_NAME", "value": "your-Docker-Hub-repo-name" }, { "name": "IMAGE_TAG", "value": "latest" } ] ...
  4. Create a build environment, run the build, and view related build information.

  5. Confirm that AWS CodeBuild successfully pushed the Docker image to the repository. Sign in to Docker Hub, go to the repository, and choose the Tags tab. The latest tag should contain a very recent Last Updated value.

Related resources

Private registry with AWS Secrets Manager sample for CodeBuild

This sample shows you how to use a Docker image that is stored in a private registry as your AWS CodeBuild runtime environment. The credentials for the private registry are stored in AWS Secrets Manager. Any private registry works with CodeBuild. This sample uses Docker Hub.

Note

Secrets are visible to actions and are not masked when written to a file.

Private registry sample requirements

To use a private registry with AWS CodeBuild, you must have the following:

  • A Secrets Manager secret that stores your Docker Hub credentials. The credentials are used to access your private repository.

    Note

    You will be charged for secrets that you create.

  • A private repository or account.

  • A CodeBuild service role IAM policy that grants access to your Secrets Manager secret.

Follow these steps to create these resources and then create a CodeBuild build project using the Docker images stored in your private registry.

Create a CodeBuild project with a private registry

  1. For information about how to create a free private repository, see Repositories on Docker Hub. You can also run the following commands in a terminal to pull an image, get its ID, and push it to a new repository.

    docker pull amazonlinux docker images amazonlinux --format {{.ID}} docker tag image-id your-username/repository-name:tag docker login docker push your-username/repository-name
  2. Follow the steps in Create an AWS Secrets Manager secret in the AWS Secrets Manager User Guide.

    1. In step 3, in Choose secret type, choose Other type of secret.

    2. In Key/value pairs, create one key-value pair for your Docker Hub user name and one key-value pair for your Docker Hub password.

    3. Continue following the steps in Create an AWS Secrets Manager secret.

    4. In step 5, on the Configure automatic rotation page, turn it off because the keys correspond to your Docker Hub credentials.

    5. Finish following the steps in Create an AWS Secrets Manager secret.

    For more information, see What is AWS Secrets Manager?

  3. When you create an AWS CodeBuild project in the console, CodeBuild attaches the required permission for you. If you use an AWS KMS key other than DefaultEncryptionKey, you must add it to the service role. For more information, see Modifying a role (console) in the IAM User Guide.

    For your service role to work with Secrets Manager, it must have, at a minimum, the secretsmanager:GetSecretValue permission.

  4. To use the console to create a project with an environment stored in a private registry, do the following while you create a project. For information, see Create a build project (console).

    Note

    If your private registry is in your VPC, it must have public internet access. CodeBuild cannot pull an image from a private IP address in a VPC.

    1. In Environment image, choose Custom image.

    2. For Environment type, choose Linux or Windows.

    3. For Image registry, choose Other registry.

    4. In External registry URL, enter the image location and in Registry credential - optional enter the ARN or name of your Secrets Manager credentials.

      Note

      If your credentials do not exist in your current Region, then you must use the ARN. You cannot use the credential name if the credentials exist in a different Region.