This post will show you how to get started with using Terraform, an open-source tool that lets you build your infrastructure using code.
What is Terraform, exactly?
Terraform is a tool designed to help you automate your cloud infrastructure. It allows you to create your infrastructure as code, using a high-level configuration language called HCL.
It is developed by HashiCorp, open-source, and licensed under Mozilla Public License 2.0.
Why Terraform?
When it comes to setting up your infrastructure, there are many considerations to be had. Almost everyone has their own unique background, and varying requirements for their systems. You might want to be geographically close to your users, or you might be operating on a tight budget. You might even want a hybrid cloud, combining resources from several cloud services.
At Redpill Linpro, we strive to accommodate our customers’ requests. Therefore, whatever the reasons, it is natural that we support the managed hosting of their infrastructure in the various clouds that are available today.
This presents us with the challenge of keeping track of the infrastructure. We definitely do not want to get into the mess of setting up everything through the various web interfaces, having no history of changes and no easy way to revert them.
Enter Terraform.
With Terraform, we can create infrastructures that are:
- Reproducible
- Scalable
- Version controlled
- Configurable by everyone in our team
How does it work?
Terraform relies on its so-called providers to interact with the API’s of the various IaaS/SaaS/PaaS services. (So you will have to install the CLI tools for the services that you want to use Terraform for.)
Each provider contains the many different resources that the service provides, such as networks, virtual servers, security groups, volumes, and so on.
There are providers for all the major players in today’s cloud environment, such as OpenStack, Amazon Web Services, Microsoft Azure, Google Cloud, and many others. You can find the complete list of providers in the documentation.
When you have created your infrastructure using code, Terraform will create an execution plan, which can then be executed in order to deploy your infrastructure.
My first Terraform configuration
In the following example, I will set up some services on Amazon Web Services.
Download the Terraform binary executable for your platform, and put the executable it in your $PATH.
Ensure that you have installed and configured the AWS CLI. (Installation and configuration of this is outside the scope of this post.)
Create a file called variables.tf:
variable "aws_region" {
description = "AWS region"
default = "eu-west-1"
}
Create another file called main.tf, and put the following code inside.
provider "aws" {
region = "${var.aws_region}"
}
Now run terraform init in order to initialize Terraform.
zcat ~/terraform-example > terraform init
Initializing provider plugins...
* Checking for available provider plugins on https://releases.hashicorp.com...
* Downloading plugin for provider "aws" (1.23.0)...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
- provider.aws: version = "~> 1.23"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
As you can see, Terraform successfully downloaded the aws provider plugin.
At the time of writing, the latest version available was 1.23.0, YMMV.
Next, let’s create an instance! We will use the AWS_instance resource for this.
Add the following lines to your code:
# main.tf
resource "aws_instance" "web" {
ami = "ami-3b261642"
instance_type = "t1.micro"
tags {
Name = "Hello World!"
}
Now we come to a crucial point of Terraform. We will generate the execution
plan by running terraform plan.
$ zcat ~/terraform-example > terraform plan
Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_instance.web
id: <computed>
ami: "ami-3b261642"
associate_public_ip_address: <computed>
availability_zone: <computed>
ebs_block_device.#: <computed>
ephemeral_block_device.#: <computed>
get_password_data: "false"
instance_state: <computed>
instance_type: "t1.micro"
ipv6_address_count: <computed>
ipv6_addresses.#: <computed>
key_name: <computed>
network_interface.#: <computed>
network_interface_id: <computed>
password_data: <computed>
placement_group: <computed>
primary_network_interface_id: <computed>
private_dns: <computed>
private_ip: <computed>
public_dns: <computed>
public_ip: <computed>
root_block_device.#: <computed>
security_groups.#: <computed>
source_dest_check: "true"
subnet_id: <computed>
tags.%: "1"
tags.Name: "Hello World!"
tenancy: <computed>
volume_tags.%: <computed>
vpc_security_group_ids.#: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.
You can consider terraform plan as a dry-run of your changes. Once you
have reviewed the execution plan above, and confirmed that your changes are as
expected, you can proceed to run terraform apply in order to have your
changes applied.
$ zcat ~/terraform-example > terraform apply
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ aws_instance.web
id: <computed>
ami: "ami-3b261642"
associate_public_ip_address: <computed>
availability_zone: <computed>
ebs_block_device.#: <computed>
ephemeral_block_device.#: <computed>
get_password_data: "false"
instance_state: <computed>
instance_type: "t1.micro"
ipv6_address_count: <computed>
ipv6_addresses.#: <computed>
key_name: <computed>
network_interface.#: <computed>
network_interface_id: <computed>
password_data: <computed>
placement_group: <computed>
primary_network_interface_id: <computed>
private_dns: <computed>
private_ip: <computed>
public_dns: <computed>
public_ip: <computed>
root_block_device.#: <computed>
security_groups.#: <computed>
source_dest_check: "true"
subnet_id: <computed>
tags.%: "1"
tags.Name: "Hello World!"
tenancy: <computed>
volume_tags.%: <computed>
vpc_security_group_ids.#: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
aws_instance.web: Creating...
ami: "" => "ami-3b261642"
associate_public_ip_address: "" => "<computed>"
availability_zone: "" => "<computed>"
ebs_block_device.#: "" => "<computed>"
ephemeral_block_device.#: "" => "<computed>"
get_password_data: "" => "false"
instance_state: "" => "<computed>"
instance_type: "" => "t1.micro"
ipv6_address_count: "" => "<computed>"
ipv6_addresses.#: "" => "<computed>"
key_name: "" => "<computed>"
network_interface.#: "" => "<computed>"
network_interface_id: "" => "<computed>"
password_data: "" => "<computed>"
placement_group: "" => "<computed>"
primary_network_interface_id: "" => "<computed>"
private_dns: "" => "<computed>"
private_ip: "" => "<computed>"
public_dns: "" => "<computed>"
public_ip: "" => "<computed>"
root_block_device.#: "" => "<computed>"
security_groups.#: "" => "<computed>"
source_dest_check: "" => "true"
subnet_id: "" => "<computed>"
tags.%: "" => "1"
tags.Name: "" => "Hello World!"
tenancy: "" => "<computed>"
volume_tags.%: "" => "<computed>"
vpc_security_group_ids.#: "" => "<computed>"
aws_instance.web: Still creating... (10s elapsed)
aws_instance.web: Still creating... (20s elapsed)
aws_instance.web: Creation complete after 23s (ID: i-04d5d6e1228b2e3f9)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Ta-da! Your new, shiny instance has been created and is now up and running.
Removing components
Maybe you would like to get rid of that instance now? You can use the
terraform destroy command for this.
zcat ~/terraform-example > terraform destroy -target=aws_instance.web
aws_instance.web: Refreshing state... (ID: i-04d5d6e1228b2e3f9)
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
- destroy
Terraform will perform the following actions:
- aws_instance.web
Plan: 0 to add, 0 to change, 1 to destroy.
Do you really want to destroy?
Terraform will destroy all your managed infrastructure, as shown above.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
aws_instance.web: Destroying... (ID: i-04d5d6e1228b2e3f9)
aws_instance.web: Still destroying... (ID: i-04d5d6e1228b2e3f9, 10s elapsed)
aws_instance.web: Still destroying... (ID: i-04d5d6e1228b2e3f9, 20s elapsed)
aws_instance.web: Still destroying... (ID: i-04d5d6e1228b2e3f9, 30s elapsed)
aws_instance.web: Still destroying... (ID: i-04d5d6e1228b2e3f9, 40s elapsed)
aws_instance.web: Destruction complete after 41s
Destroy complete! Resources: 1 destroyed.
The observant reader will have noticed I used the command
terraform destroy -target=aws_instance.web, explicitly specifying the
name of the resource I want to destroy. I find this to be a good habit, as
running terraform destroy without any parameters can have disastrous
consequences.
If you want to create the instance again, just run terraform plan and
terraform apply.
Enabling remote backends with state locking
Terraform has support for backends, which is a way of storing the current
state. The default backend is local, which means the state will be saved in
the Terraform working directory, in a file called terraform.tfstate.
But assuming you work in a team, having the current state available locally is no good, unless your team members are also able to access it.
Sure, you could commit the terraform.tfstate file between every state
change, but that would be a hassle, and prone to errors, as people will
undoubtedly forget to push/pull the latest changes.
In this example, we will be changing the backend to an S3 bucket. We will also create a DynamoDB table, in order to implement state locking and consistency checking of our Terraform state.
First, we will create the bucket (using Terraform, of course!) by adding the following code:
# main.tf
resource "aws_s3_bucket" "tfstatebucket" {
bucket = "rl.tfstate"
acl = "private"
# We want to have versioning enabled, because it allows us to keep track of
# the Terraform state history
versioning {
enabled = true
}
# We also want to make sure our bucket enables server-side encryption
server_side_encryption_configuration {
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
tags {
Name = "Terraform state bucket"
}
}
resource "aws_dynamodb_table" "tfstatedynamotable" {
name = "rl.tfstate"
hash_key = "LockID"
read_capacity = 5
write_capacity = 5
attribute {
name = "LockID"
type = "S"
}
tags {
Name = "Terraform state consistency"
}
}
Run terraform plan and terraform apply to create the bucket and the
DynamoDB table.
Now that we have our S3 bucket and our DynamoDB table, we can change the Terraform backend by adding the following code:
# main.tf
terraform {
backend "s3" {
bucket = "rl.tfstate"
key = "terraform.state"
region = "eu-west-1"
encrypt = true
dynamodb_table = "rl.tfstate"
}
}
Okay, now all we have to do is to initialize our Terraform configuration again, this time with our new backend.
zcat ~/terraform-example > terraform init
Initializing the backend...
Do you want to copy existing state to the new backend?
Pre-existing state was found while migrating the previous "local" backend to the
newly configured "s3" backend. No existing state was found in the newly
configured "s3" backend. Do you want to copy this state to the new "s3"
backend? Enter "yes" to copy and "no" to start with an empty state.
Enter a value: yes
Releasing state lock. This may take a few moments...
Successfully configured the backend "s3"! Terraform will automatically
use this backend unless the backend configuration changes.
Initializing provider plugins...
The following providers do not have any version constraints in configuration,
so the latest version was installed.
To prevent automatic upgrades to new major versions that may contain breaking
changes, it is recommended to add version = "..." constraints to the
corresponding provider blocks in configuration, with the constraint strings
suggested below.
* provider.aws: version = "~> 1.23"
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
There you go! The Terraform state is now stored in the S3 bucket we created.
Create a VCS repository with the files you have created in this article, and
share it with your team in order for them to get started using Terraform too.
Just remember to remove the terraform.tfstate and
terraform.tfstate.backup files from your working directory!
That’s all for now, folks
This was a basic introduction to Terraform, which hopefully will get you started on the right track.
The documentation will be your best friend when writing HCL. There are also many examples of Terraform code that you can find online, and use as a starting point for your own infrastructure.
In my next article, I will be introducing you to the concept of modules in Terraform, which allows us to re-use Terraform configurations and keep our code DRY.