Menu
AWS Service Catalog Validation Pipeline
AWS Service Catalog Validation Pipeline

Implementation Considerations

Test Functions

The AWS Service Catalog Validation Pipeline includes a set of preconfigured AWS Lambda functions for validating product template code (pre-create tests) and test product stacks (post-create tests). These functions are part of the Central Microservices stack, which you launch using the solution-provided AWS CloudFormation template.

The following table describes each function and the test it runs.

Lambda Function Test Type Description
Validate_Template Pre-create, Default Runs a native AWS CloudFormation template validation command that checks the syntax of your template.

Note

This Lambda function is hardcoded into the pipeline.

Lint_Template Pre-create, Optional Builds the template in AWS CodeBuild and runs predefined cfn-nag rules to validate the code. Cfn-nag is a third-party linting tool for AWS CloudFormation templates. For more information and troubleshooting guidance, see Appendix A.

Note

If you choose to run this test, the solution will deploy AWS CodeBuild.

Subnet_Name Post-create, Optional Runs a test to search for specific string values in VPC subnet name tags. This function is provided as a starting point for integrating tag-based string checks into your pipeline.
Test_Connectivity Post-create, Optional For templates that deploy networking resources, tests for outbound network connectivity. The test searches for private subnets by name, and, in each subnet, launches a Lambda function that attempts to connect to an endpoint. The test searches for subnet names that begin with PrivateSubnet, but you can modify the test (for example, name string and endpoint) as necessary.
AMI_Check Post-create, Optional Verifies that any Amazon Machine Images (AMIs) references in the template are up to date.

The solution includes these common tests as a starting point. Customers can use these Lambda functions as a reference for creating additional unit tests tailored to their requirements. For example, you can test a template against the AWS Well-Architected framework or rank a template according to the Center for Information Security (CIS) AWS Foundations Benchmark. See Appendix B for detailed instructions.

The solution’s AWS CloudFormation template includes parameters for four pre-create tests and four post-create tests. You modify the AWS CodePipeline resource in the solution template to increase the number of tests. During initial configuration, enter the exact names of the Lambda functions (solution-provided or custom) to include those tests in your validation pipeline.

Configuration Files

The AWS Service Catalog Validation Pipeline uses customer-managed parameter files and a configuration file (config.yml) to launch test product stacks. The parameter files supply default values for a product’s test stack. The configuration file specifies the templates to validate, the parameter files to use for each test stack, and the AWS Regions to validate those test stacks in. This methodology is based on the AWS Quick Start framework for automated testing. For more information, see Create a parameters file for automated testing. For instructions on preparing these files, see Step 1.

These files are stored in the ci folder in your repository (see Repository Requirements). As explained in the previous sections, during initial configuration, you specify which tests (Lambda functions) to include in your pipeline. The pipeline will run the pre-create tests on your product template(s). If your pre-create tests include calls to the AWS SDK, these calls will default to the same AWS Region(s) you have designated the test product stack(s) to run in. The pipeline will then create a temporary AWS Service Catalog portfolio and provision test products as specified in the configuration file, and run post-create tests on those products.

Test Stack Timeouts

The product provisioning stage is a custom action, and is subject to an AWS CodePipeline timeout limit of one hour. If your test product stacks collectively take longer than one hour to launch, the stage will fail and return a timeout error. To work around this issue, break your template into smaller substacks and test them in separate pipelines.

Repository Requirements

The AWS Service Catalog Validation Pipeline is designed to integrate with an existing AWS CodeCommit repository. To launch the solution successfully, your repository must contain two folders: template and ci. The ci folder contains the config.yml file described in the previous section.

If you want to use an Amazon Simple Storage Service (Amazon S3) bucket or GitHub as your repository location, you must modify the source stage of the pipeline and configure access appropriately. See Working with Pipelines in AWS CodePipeline for detailed instructions. For guidance on incorporating Git repositories hosted elsewhere, see Integrating Git with AWS CodePipeline on the AWS DevOps Blog.

Regional Restrictions

You must deploy this solution in an AWS Region that supports AWS Service Catalog, AWS CodePipeline, AWS CodeBuild, and AWS CodeCommit (see the AWS website for the most current service availability by region). Once deployed, the solution is designed to validate products in any AWS Region that supports AWS Service Catalog. The pipeline will provision test product stacks and run post-create tests in the region(s) you specify in your test configuration (config.yml) file.

Continuous Delivery

This solution creates a validation pipeline that is scoped for continuous integration of AWS CloudFormation templates hosted in an AWS CodeCommit repository. This includes automated logical and function testing of AWS Service Catalog products, and a manual approval stage to then copy the validated product template to an Amazon S3 bucket. This solution is not designed to deliver product updates to an AWS Service Catalog portfolio, although we recommend integrating a continuous delivery solution to manage your product versions. See Building a Continuous Delivery Pipeline for AWS Service Catalog on the AWS DevOps Blog for an example of a solution for continuous delivery of product templates from an AWS CodeCommit repository to an AWS Service Catalog portfolio.