Building a CloudFormation Template for Virtual Private Cloud (VPC)

In a few months or even weeks, CDK will probably take over the CloudFormation success. So I have to ship this blog post now or never. Over the last decade, I have built a lot of VPC templates, but recently I discovered that this one contains a lot of CloudFormation learnings as well. This blog post describes how to create a VPC template full of easy to learn CloudFormation features. Enjoy!

Prerequisites

You should have some basic knowledge about Networking. So terms like Routing, Subnetting, IP Addresses, should not really scare you off. However, even with only a tiny bit of knowledge and IT background, I think it should be able to follow this blog post. I’m not going to explain all the terms and calculations.

Purpose

The end result template is very useful for POCs, Prototypes and AWS Learning Exercises. You’ll learn:

• How to use Parameters, Conditions, Mappings, Resources and Outputs in CloudFormation
• How to use CloudFormation Intrinsic Functions like: !Ref, !GetAtt, !FindInMap, !Sub, !Select, !GetAZs, !Cidr, !Or, !Equal (80% of all available functions)
• Some opiniated best practices / clean code in CloudFormation templates

Limitations

It’s not my idea to create one template that fits all use cases. Hence, there are some limitations to be aware of:

• Maximum of 3 Availability Zones (there are only 1 or 2 regions that support 3+ AZs)
• It is always a 3 layered design: Public, Private and Isolated
• NACLs, Gateways and Route Tables are left out in the blog post. Gateways and Route Tables are part of the templates you could download at the end.

Design

Before I dive into code, first a diagram of the end result.

Parameters

AWS is very flexible, so you can make every network design possible. However, more than 90% of the templates look identical. They are layered, with public, private and isolated layers and distributed across 1,2,3 or more AZs. They are either large, a /16 Cidr Block VPC with /20 Subnets or very small, with a /20 VPC and /24 Subnets. If you mix these flavours, you get a small list.

Goal: I want to specify the VPC CidrBlock, NetworkSize (small, medium or large) and the number of AZs to use, and everything else is generated based on these input parameters.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

Parameters:

  VPCCidrBlock:
    Type: String
    Default: "10.0.0.0/16"

  NetworkSize:
    Type: String
    Default: "Large"
    AllowedValues: 
      - "Large"
      - "Medium"
      - "Small"

  AZs:
    Type: String
    Default: 3
    AllowedValues: [1, 2, 3]

Mappings

When I’m defining resources, I often need to select an “index” from a list. This index is a number, which is different per list. For example:

1
2

# returns "bananas" because it's the 2nd index
!Select [ 2, [ "oranges", "apples", "bananas" ] ] 

I created this list of “global” properties in the section Mappings. I try to use Mappings only for values that don’t change often. I carefully chose the naming, refactored it during the creation of the template, to make the template easy to read. More on that later.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52

Mappings:

  NetworkLayout:
    Public:
      a: 0
      b: 1
      c: 2
    Private:
      a: 3
      b: 4
      c: 5
    Isolated:
      a: 6
      b: 7
      c: 8

  NetworkConfig:
    Large:
      SubnetCidrCount: 16
      SubnetCidrBits: 12
    Medium:
      SubnetCidrCount: 16
      SubnetCidrBits: 10
    Small:
      SubnetCidrCount: 16
      SubnetCidrBits: 8
  
  Index:
    Layer:
      Public: 0
      Private: 1
      Isolated: 2
    AZ:
      a: 0
      b: 1
      c: 2

  Letter:
    Upper:
      a: "A"
      b: "B"
      c: "C"
      A: "A"
      B: "B"
      C: "C"
    Lower:
      a: "a"
      b: "b"
      c: "c"
      A: "a"
      B: "b"
      C: "c"

Conditions

In the parameter AZs I specified how many AZs I want to use between 1 and 3. The default is 3. The conditions ensures a resource is created or not. It looks a bit silly, like I don’t understand how to work with GreaterThen or LowerThen functions. Again, these functions don’t exist in CloudFormation. !Equals does…

1
2
3
4
5
6
7
8

Conditions:
  CreateResourcesInAZA: !Or
    - !Equals [ !Ref AZs, 1 ]
    - !Condition CreateResourcesInAZB
  CreateResourcesInAZB: !Or
    - !Equals [ !Ref AZs, 2 ]
    - !Condition CreateResourcesInAZC
  CreateResourcesInAZC: !Equals [ !Ref AZs, 3 ]

Template

Finally the template. In the next section you can download the complete template, with 15 Subnets. In this example there is only one:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

Resources:

  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VPCCidrBlock

   # every line that contains "<--" must be changed for each resource
   # with the Letter Upper Mapping, I can always enter Upper for consistency

  SubnetPrivateA: # <-- Public, Private, Isolated & A, B, C
    Type: AWS::EC2::Subnet
    Condition: CreateResourcesInAZA # <-- A, B, C
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: 
        !Select 
        - !FindInMap
          - "Index"
          - "AZ"
          - !FindInMap ["Letter", "Lower", "A"] # <-- A, B, C
        - !GetAZs ""
      CidrBlock:
        !Select
        - !FindInMap
          - "NetworkLayout"
          - "Private" # <-- Public, Private, Isolated
          - !FindInMap ["Letter", "Lower", "A"] # <-- A, B, C
        - !Cidr
          - !GetAtt VPC.CidrBlock
          - !FindInMap ["NetworkConfig", !Ref NetworkSize, "SubnetCidrCount"]
          - !FindInMap ["NetworkConfig", !Ref NetworkSize, "SubnetCidrBits"]
      Tags:
        - Key: "Name"
          Value:
            !Sub
            - "${stack}-${layer}-${az}"
            - stack: !Ref 'AWS::StackName'
              layer: "Private" # <-- Public, Private, Isolated
              az: !FindInMap ["Letter", "Upper", "A"] # <-- A, B, C

When you need to create 15 copies of this Subnet, and other resources like RouteTables etc, you’ll face one of the ugly things of CloudFormation. There is a lot of code duplication. To automate this, a templating engine like Jinja could help you out. Just to generate the final CloudFormation template without typos and much quicker.

1
2
3
4
5
6
7
8
9

{%- for layer in ['Public', 'Private', 'Isolated'] %}
{%- for az in ['A', 'B', 'C'] %}

# Subnets {{ layer }} {{ az }}

# CloudFormation Resources

{%- endfor %}{#end of AZs loop#}
{%- endfor %}{#end of Layer loop#}

1
2
3
4
5
6

$ pip install jinja2
$ jinja2 template.jn2 > template.yml
$ aws cloudformation deploy \
  --stack-name vpc \
  --template template.yml
...

Conclusion

We have learned how to build a CloudFormation VPC Template that is easy to re-use, together with some CloudFormation tips & tricks.

All code snippets in this blog post combined, including Gateway, Route Tables and some documentation is also available for download. What I also did and might be a tip. With a tag ‘Name’ most resources get a name that is easier to recognize. In this template, all resources are using the format: “${AWS::StackName}-LogicalId”. So if in the template a resource has a Logical ID: SubnetPrivateA and the CloudFormation stack is called: “VPC”, the resource is called: “VPC-SubnetPrivateA”.

Warning! If you try to deploy 3 AZs in a Region that only offers 2 AZs, you’ll run in to the following error. This is becausae the !GetAZs returns a list of 2 Availability Zones (index 0 and 1) and you try to select number 2.

1

Template error: Fn::Select cannot select nonexistent value at index 2

A small update. Somehow I couldn’t sleep last night due to the fact I use 3 parameters. I was wondering how the template looks like if I only have 2 parameters: The VPC Cidr Block, and one parameter that describes how many Availability Zones to use (1az, 2az or 3az) AND the size: small, medium or large. So here they are: template-v2.jn2 and template-v2.yml. While working on this new template, I event recognized how I could optime the orignal template.

-Martijn

Photo by Dawid davealmine on Unsplash