Fostering Cross Account AWS CodePipelines

AWS automation recently got a boost: the company introduced the ability to build an end-to-end release automation workflow that can deploy changes across multiple regions or different AWS accounts. And they subsequently featured an article on their blog on the steps to create a cross region CodePipeline. Today, however, we want to address the other half of this equation — building cross account pipelines — and thought it worthwhile to share with you here when and why we would recommend the benefits of this approach.

Separate Yet Central

For many companies, especially large enterprises, it is imperative to maintain separate AWS accounts. This separation of accounts provides much needed resource isolation for administrative, security and audit reasons. This often manifests itself in the form of separate accounts for production, development, QA, etc. While this separation is much needed (and often mandated within the organization), these groups naturally would still like to have a central pipeline for continuous integration and continuous delivery (CI/CD). And in these cases Flux7 consultants recommend building a cross account code pipeline with AWS CopePipeline.

Benefits of a Central CI/CD

A central pipeline for CI/CD helps enterprises in several important ways. And, AWS Code Pipeline is the recommended AWS solution as it is a managed service. CodePipeline builds, tests, and deploys code every time there is a code change, based on the release process model(s) defined by the organization. This enables enterprises to rapidly and reliably deliver features and updates.  

A central CI/CD across AWS accounts is beneficial because it:

• Helps with the build once, deploy multiple times strategy. An ‘oldie but goodie’, this strategy is invaluable in that it simplifies deployment, making it repeatable and much easier to identify errors or issues in the process.
• Keeps all the build artifacts in a single place. In the case of AWS CodePipeline, the S3 bucket where the code pipeline is created serves as the repository for these artifacts.

Moreover, at Flux7 we have developed an internal project where we have a code pipeline to deploy AWS Config-rules — which help in audit and compliance checks — across multiple accounts using custom AWS Lambda functions. And, with custom Lambda functions, these pipelines can be used for many other applications like compliance checks, monitoring, and more. CodePipelines can also be used to create/update AWS CloudFormation stacks. So infrastructure changes across multiple environments/accounts can be easily orchestrated.

Cross Account Pipelines at Work

We recently worked with a Fortune 500 manufacturer of heavy equipment that has more than 50 AWS accounts. One of the more interesting aspects of our work with them is the deployment pipeline we created. The pipeline is implemented using AWS CodePipeline with the code for the Lambda functions stored in an on-premise GitHub enterprise server. CodePipeline is triggered when a change to code is made. To ease deployments, updates, and rollbacks, we use AWS CloudFormation for its built-in features; we set up each of the Lambda functions as their own CloudFormation stack and wrap them as nested stacks in an orchestration stack.

This top-level nested CloudFormation template is auto-generated by a Python script. The script reads out the directories for the code for each of the AWS Lambda functions and uses the directory names to create the template. The process for adding a new notification is quite simple: create a new directory and write the code for it in that folder. The Python scripts adds it to the template automatically and CodePipeline deploys it. All this with a single code pipeline where the artifacts remain in the related AWS CodePipeline S3 bucket.

For additional reading on AWS CodePipeline and effectively managing AWS activities across accounts:

• How to Create a Pipeline in AWS CodePipeline That Uses Resources from Another AWS Account
• AWS CodePipeline, CloudFormation & Continuous Delivery
• Amazon Pipeline Starter Kit for Continuous Delivery
• AWS Cross Accounts Access: Defining the Best AWS Account Architecture
• AWS Cross Accounts Access Set-Up and Benefits

As AWS consultants who see the benefits of CI/CD, we appreciate features like cross account code pipelines that enable organizations to more quickly reap the benefits of cloud-based DevOps. To read more about optimized code delivery click here to download a paper or for ongoing tips, tricks and analysis, please subscribe to our blog below.