Provisioning AWS CodePipeline using Terraform
Terraform is an open source Infrastructure as Code (IaC) tool that enables you to provision, iterate upon, and manage infrastructure by using Hashicorp Configuration Language (HCL). It can provision and manage infrastructure from multiple cloud providers as well as custom in-house solutions.
AWS CodePipeline is a fully managed AWS service that automates the build (CodeBuild), test (CodeBuild), and deploy (CodeDeploy) phases of your release process every time there is a code change. It is similar to tools such as CircleCI. The beauty of CodePipeline is that it can integrate with most of the tools that are popular in a DevOps focused environment. It can pull your source code from S3, GitHub, or CodeCommit. It can run builds and unit tests using a yaml templated document called a buildspec in CodeBuild, and then deploy your changes to CloudFormation, CodeDeploy, S3, or any custom resource you configure/specify.
Since Terraform is my go IaC tool and with the recent release of Terraform v0.12.0, I wanted to share a sample pipeline built using Terraform. The included module creates a pipeline that pulls from S3, build using CodeBuild to create 3 simple Lambda Functions written in Python, and deploys the output artifact to S3. The pipeline is triggered every time there is a push/upload to the S3 Bucket. The pipeline also includes a manual approval step just as an example to show some of the features of CodePipeline. Once the plan is approved by entering a comment on the CodePipeline, the rest of the pipeline steps are automatically triggered.
Running Terraform Locally
Usually, in the case of a CI/CD Pipeline, you would have a way to trigger the terraform steps below by using an automation server such as Jenkins that would trigger builds and testing on merges to the master branch of your source code repository. In this example, I will be running the terraform commands locally.
Note: The versions.tf file forces the use of Terraform v0.12.0
Clone from my GitHub repository
git clone https://github.com/PatriciaAnong/CodePipeline.gitcd CodePipelineConfigUpdate the terraform.tfvars file with the appropriate parameters and then initialize the working directory, generate a plan, and apply.
After you initialize
terraform init
You could choose to plan and apply each module separately by specifying a target: terraform plan -target module.prereqs
or you could just run a plan from the root module to create resources in all the child modules.
terraform plan
To ensure the configuration outputted by the plan is what is created, you would generate a plan and specify an out file
terraform plan -out PAnongExample
If the plan is generated correctly and without errors, create the resources by running the apply command using the out file from the plan command:
terraform apply PAnongExample -auto-approve
Once the apply is complete - you will have a CodePipeline with two CodeBuild projects. They will be triggered automatically and require a manual approval step - a SMS notification will be sent to the approver (whatever phone number you entered for the subscriptions variable), that person would then go the CodePipeline Console and either approve or reject the plan created in the last step. If it is approved, the rest of the pipeline steps proceed automatically. Once the pipeline is done, you will have 3 Lambda functions.
*Cover Images from buildo.io