Starting with CloudFormation templates

This is not the place to tell anyone why Infrastructure as Code is a good idea. For that I can point the potential readers to a blog by my colleague Yngve about that: Why code your infrastructure?

I a short series of blogs, I intend to demonstrate building infrastructure in AWS in steps, where I will be building upon previous entries. Basic knowledge of network and VPC is assumed.

Note that following these instructions can and will incur costs from AWS, those are the sole responsibility of the user, not me.

What is CloudFormation?

CloudFormation is AWS way of doing Infrastructure as Code. The templates can be written in YAML or JSON. We will only look at the YAML format as that is the one that is actually human readable.

Resource for information

The AWS documentation on CloudFormation is outstanding. I do not try to construct anything in AWS without the guidance of the documentation. As a side note, the navigation in the AWS documentation is terrible, use a search engine as DuckDuckGo or Google (the brave might want to try Bing) to find what you are looking for.

The term stack is used a lot here. We call the collection of resources created from the templates a stack.

Anatomy of a template

The template can consists of following sections

  • Description
  • Metadata
  • Parameters
  • Mappings
  • Conditions
  • Resources
  • Outputs

Of these, only the Resources section is required.

Description

Just a free text that can be used as a description for your stack.

Metadata

You can use the optional Metadata section to include arbitrary JSON or YAML objects that provide details about the template. I have yet to be bothered.

Parameters

Variables to your stack, can be used to build different environments depending on the values. Values of the individual parameters can be defined in an external JSON structure.

Mappings

Used for different sets of values for a specific key. So you can have a list of AMIs for each AWS Region, or set of different EC2 instance types depending on whether the environment should be stage or production.

Conditions

Here you can define boolean variables that can be used as Condition field in the Resources section

Resources

This is the main part. Here all the definition of the environment is done.

Outputs

Here you can define and format values that are easily accessible from querying your stack, and also define values that are exported from your stack and can be imported into other stacks. Each export from a stack needs to have an unique name within a Region (within your AWS account)

Regarding tags

For this write up, I gloss over the usage of tags, in my opinion one can do that in the learning phase, but tags are really useful assets to use. So I would encourage users to spend some time designing a tag regime for their resources, and manage those through CloudFormation.

Starting on your first stack

Let us create a VPC. If you have not done this before, I highly recommend you to start by looking at the AWS documentation at AWS::EC2::VPC

From the section Syntax we get the following YAML code:

---
Type: AWS::EC2::VPC
Properties:
  CidrBlock: String
  EnableDnsHostnames: Boolean
  EnableDnsSupport: Boolean
  InstanceTenancy: String
  Tags:
    - Tag

If we then read on the Properties section, we see that the only thing that is required is the CidrBlock, so by stripping all other entries from the YAML we end up with this minimal CloudFormation template

Small steps

---
Description: This is an attempt to create a VPC in a Cloudformation stack
Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16

If we save this to a file called vpc.yaml then we can create a VPC with a command like this, given we have a CLI profile called blogg (more about AWS CLI and profiles here :

$ aws --profile blogg cloudformation create-stack --stack-name myawesomeVPC --template-body file://vpc.yaml
{
    "StackId": "arn:aws:cloudformation:eu-central-1:123NaN123201:stack/myawesomeVPC/b9fe6cab-9dfc-4839-b5eb-16bed311730c"
}

After a short while the stack has been created and the VPC has come to existence

$ aws --profile blogg cloudformation describe-stacks --stack-name myawesomeVPC
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:eu-central-1:123NaN123201:stack/myawesomeVPC/b9fe6cab-9dfc-4839-b5eb-16bed311730c",
            "StackName": "myawesomeVPC",
            "Description": "This is an attempt to create a VPC in a Cloudformation stack",
            "CreationTime": "2020-02-30T25:66:42.898Z",
            "RollbackConfiguration": {},
            "StackStatus": "CREATE_COMPLETE",
            "DisableRollback": false,
            "NotificationARNs": [],
            "Tags": [],
            "EnableTerminationProtection": false,
            "DriftInformation": {
                "StackDriftStatus": "NOT_CHECKED"
            }
        }
    ]
}

Next step

As this is rather useless as it is, let us add a sub-net, we check the documentation on the AWS website and get the following YAML for a sub-net definition

Type: AWS::EC2::Subnet
Properties:
  AssignIpv6AddressOnCreation: Boolean
  AvailabilityZone: String
  CidrBlock: String
  Ipv6CidrBlock: String
  MapPublicIpOnLaunch: Boolean
  Tags:
    - Tag
  VpcId: String

The only properties values that must be set are VpcId and CidrBlock. The value of VpcId comes from the Resource created before, and CidrBlock needs to be a CIDR inside the VPC CIDR.

So with that we modify the old YAML structure to

---
Description: This is an attempt to create a VPC in a Cloudformation stack
Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16

  SubNett:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: 10.0.42.0/24
      VpcId:
        Fn::Ref: VPC

Note that we use the intrinsic function Fn::Ref there. In this case it returns the value of the AWS::EC2::VPC resource, so we do not need to hard code it into the template. Read more about the intrinsic function here

We can then update our first stack to with those changes

$ aws --profile blogg cloudformation update-stack --stack-name myawesomeVPC --template-body file://vpc.yaml
{
    "StackId": "arn:aws:cloudformation:eu-central-1:123NaN123201:stack/myawesomeVPC/19cb1c41-b3cb-4a1b-9d75-aef7df5fddb4"
}

What we would then have is a VPC with a sub-net with no access to anything. It could be used for EC2 instances talking to each other, but that would be about all that you could do. In other words, not that useful.

Internet connectivity

So what is needed to have connectivity to the Internet is an Internet Gateway, That is AWS::EC2::InternetGateway in CloudFormation. If we read the documentation for it is only

Type: AWS::EC2::InternetGateway
Properties:
  Tags:
      - Tag

and the Tags element is not required, so let us add that to our template.

---
Description: This is an attempt to create a VPC in a Cloudformation stack
Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16

  SubNett:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: 10.0.42.0/24
      VpcId:
        Fn::Ref: VPC

  InternetGW:
    Type: AWS::EC2::InternetGateway

But that gateway is not connected to our VPC at all. For that we need VPC Gateway Attachment, which is defined with AWS::EC2::VPCGatewayAttachment and is described in the documentation as

Type: AWS::EC2::VPCGatewayAttachment
Properties:
  InternetGatewayId: String
  VpcId: String
  VpnGatewayId: String

where we must declare one and only one of InternetGatewayId or VpnGatewayId, we are not dealing with a VPN, so our entry is InternetGatewayId. Adding to our main template it becomes

---
Description: This is an attempt to create a VPC in a Cloudformation stack
Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16

  SubNett:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: 10.0.42.0/24
      VpcId: !Ref VPC

  InternetGW:
    Type: AWS::EC2::InternetGateway

  GatewayToInternet:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGW

Notice that we have now gone over to the short notation of the intrinsic function Fn::Ref which is just !Ref. There is a limitation on the short notation that you can not chain them, but that is a worry for another time.

The sub-net that would be managed with this template is one still without Internet connection, as there is no routing there yet.

Routing

We first need a route table, then a route in that route table, and then we need to attach the route table to our sub-net. In order, the elements we need are: AWS::EC2::RouteTable, AWS::EC2::Route and AWS::EC2::SubnetRouteTableAssociation

The documentation for those elements gives us

Type: AWS::EC2::RouteTable
Properties:
  Tags:
    - Tag
  VpcId: String

Here, the tags are optional.

Type: AWS::EC2::Route
Properties:
  DestinationCidrBlock: String
  DestinationIpv6CidrBlock: String
  EgressOnlyInternetGatewayId: String
  GatewayId: String
  InstanceId: String
  NatGatewayId: String
  NetworkInterfaceId: String
  RouteTableId: String
  TransitGatewayId: String
  VpcPeeringConnectionId: String

Here the RouteTableId is mandatory, and we want to connect it to our gateway so we use the GatewayId element.

Also the DestinationCidrBlock is mandatory (or the DestinationIpv6CidrBlock if you are defining IPv6 network). Here we would have the destination 0.0.0.0/0, AKA everything. So all traffic not going to the IP range of the VPC will be sent to that route.

Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
  RouteTableId: String
  SubnetId: String

Both are required.

So when we combine all this, we get:

---
Description: This is an attempt to create a VPC in a Cloudformation stack
Resources:
  VPC:
    Type: 'AWS::EC2::VPC'
    Properties:
      CidrBlock: 10.0.0.0/16

  SubNett:
    Type: AWS::EC2::Subnet
    Properties:
      CidrBlock: 10.0.42.0/24
      VpcId: !Ref VPC

  InternetGW:
    Type: AWS::EC2::InternetGateway

  GatewayToInternet:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGW

  RouteTablePublic:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC

  RoutePublicIPv4:
    Type: AWS::EC2::Route
    Properties:
      DestinationCidrBlock: 0.0.0.0/0
      RouteTableId: !Ref RouteTablePublic
      GatewayId: !Ref InternetGW

  Route2Subnet:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref RouteTablePublic
      SubnetId: !Ref SubNett

And with that template, we have a functioning VPC with a sub-net that can connect to the internet. In later blogs I will build upon this template to create a production ready VPC, and then build environments in those VPCs. Stay tuned.

Jónas Helgi Pálsson

Senior Systems Consultant at Redpill Linpro

Jónas joined Redpill Linpro over a decade ago and has in that period worked as both a consultant and a system administrator. Main focus currently for Jónas is AWS and infrastructure on that platform. Previously been working with KVM and OpenStack, dabbles with programming and has a soft spot for openSUSE.

More Posts

Thoughts on the CrowdStrike Outage

Unless you’ve been living under a rock, you probably know that last Friday a global crash of computer systems caused by ‘CrowdStrike’ led to widespread chaos and mayhem: flights were cancelled, shops closed their doors, even some hospitals and pharmacies were affected. When things like this happen, I first have a smug feeling “this would never happen at our place”, then I start thinking. Could it?

Broken Software Updates

Our department do take responsibility for keeping quite a lot ... [continue reading]

Alarms made right

Published on June 27, 2024

Just-Make-toolbox

Published on March 22, 2024