Be a Terraform Ninja Using Jinja

Infrastructure as code (IaC) is the process of managing and provisioning computer infrastructure through a programming language and infrastructure definition files instead of through the manual process of hardware configuration. Terraform is a tool for building and versioning infrastructure as code, developed by Hashi Corp. It allows users to define infrastructure as code in high-level language scripting called HCL to safely and predictably create, change and improve infrastructure and maintain the infrastructure code in a repository which can be versioned and audited.

HCL is great for many projects due to its simplicity, but it lacks the flexibility and feature sets offered by more full-featured languages, like Java, python or javascript. To overcome some of the limitations of the Terraform scripting language, we can use a template engine like Jinja to wrap an HCL script using template directives and generate the HCL script dynamically and execute the resulting code.

Lets walk through a simple example that illustrates the advantages of using a template engine. Say you need to create several AWS EC2 instances, each with a different set of configuration parameters provided in an input variable as an array of maps.

Using terraform HCL, you can create multiple EC2 instances using an “aws_instance” resource with “count” attribute set to the size of the array to iterate through the list of instance configuration settings in the array and create the instances. In terraform v11, creating instances using “count” poses a major problem when the instances are being destroyed/modified. If any changes are made to the configuration of any instance that required instance recreation, it will destroy and recreate not only the specific instance, but all instances created after the modified instance will also be destroyed and recreated.

To overcome this problem, we can use Jinja template engine directives to wrap around HCL code and generate terraform code dynamically. Since each resource in the generated terraform code is independent, this approach solves the problem and allows individual resource configuration changes without impacting other EC2 instances.

Sample Code
The following terraform.tfvars file contains EC2 instance configurations specified in a list of maps, to spin up linux and windows servers each with different configuration parameters and startup scripts.

Below is sample terraform script using count.

If the above script is executed, it will create the number of EC2 instances with specified configurations in the terraform variable “instance_list”. If you make any changes to the configuration of a specific instance after the instances were created, terraform will destroy and recreate all the instances after the specific instance.

To solve the problem, we can use Jinja template wrapper to generate terraform code specific to each instance, as shown below:

1. Once the script is wrapped using template directives, terraform code can be generated using python script render.py.

2. The script will render output terraform files in the folder, each instance will have a separate resource section in the output script which can be run to spin up the instances. Any changes to an individual instance configuration will not impact other instances since each resource is created separately.

Granted, in terraform V12, this issue can be overcome using the “foreach” keyword while creating the resources which solve this issue. However, there are many other situations where Jinja template directives can be used to power terraform code and implement dynamic behavior that cannot be accomplished using the terraform scripting language of HCL.

Do you have any projects or installations that might benefit from our experience with IaC, Terraform, Jinja, or any other automation technologies? We at Anexinet would love to have a conversation to see how we can help. Please don’t hesitate to reach out to us.