Skip to content
Tutorials Center
My learning

Topics

Lesson 52 of 70 ยท Infrastructure as code

CloudFormation

Learn AWS CloudFormation: templates, stacks, parameters, outputs, change sets, drift detection and nested stacks, with a runnable Ref and Sub template resolver.

Clicking through the console is fine for learning. It is a poor way to run a business: nobody can say exactly what was clicked, you cannot repeat it in a second account, and a reviewer cannot read it before it goes live. Infrastructure as code replaces clicks with a text file that describes your infrastructure, and you keep that file in version control like any other code.

AWS CloudFormation is AWS's own service for this. You write a template (YAML or JSON) that lists resources and their properties. CloudFormation turns it into a stack: a group of real resources that it creates, updates and deletes together. This lesson covers the template language, the stack lifecycle and the safety tools (change sets and drift detection). The next lessons show two other tools that build on or compete with it: AWS CDK and Terraform on AWS.

CloudFormation needs a real AWS account, so templates and CLI commands here are read-only. The runnable examples are Python stand-ins that resolve Ref and Fn::Sub in a template, order resources by their dependencies, preview a change set and detect drift. They teach the idea and omit many real features, and each section says what is simplified.

Templates

A template has a few top-level sections. Only Resources is required.

YAML
AWSTemplateFormatVersion: "2010-09-09"
Description: A logs bucket and an app bucket that writes access logs to it

Parameters:
  Environment:
    Type: String
    AllowedValues: [dev, prod]
    Default: dev

Conditions:
  IsProd: !Equals [!Ref Environment, prod]

Resources:
  LogBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Retain

  AppBucket:
    Type: AWS::S3::Bucket
    Properties:
      VersioningConfiguration:
        Status: Enabled
      LoggingConfiguration:
        DestinationBucketName: !Ref LogBucket

  ProdAlarmTopic:
    Type: AWS::SNS::Topic
    Condition: IsProd

Outputs:
  AppBucketName:
    Description: Name of the application bucket
    Value: !Ref AppBucket
    Export:
      Name: !Sub "${AWS::StackName}-app-bucket"

What each section does:

  • Parameters are inputs you supply when you create or update the stack, such as an environment name or an instance type. They let one template serve many environments.
  • Mappings are fixed lookup tables (for example an AMI ID per Region).
  • Conditions are true/false values computed from parameters. A resource with a Condition is created only when the condition is true.
  • Resources is the heart of the template. Each entry has a logical ID (AppBucket), a Type such as AWS::S3::Bucket and Properties. The real thing that CloudFormation creates gets a physical ID, such as the actual bucket name.
  • Outputs publish values: shown on the stack, and optionally exported for other stacks.
  • Other sections: Metadata, Rules and Transform (which invokes macros such as AWS SAM for serverless shortcuts).

Notice that AppBucket never says "create LogBucket first". The !Ref LogBucket already tells CloudFormation: it builds a dependency graph from the references and creates things in a valid order, with independent resources in parallel. You can add DependsOn for dependencies that no reference expresses.

Intrinsic functions

Values inside a template are computed by intrinsic functions. The short YAML forms start with !.

Function Meaning
!Ref Name A parameter's value, or a resource's main identifier (a bucket's name, a security group's ID, a topic's ARN)
!GetAtt Res.Attr A named attribute of a resource, such as AppBucket.Arn
!Sub "text ${Var}" Fill variables into a string; ${Res.Attr} works like GetAtt
!Join [sep, [a, b]] Join a list into a string
!If [Cond, a, b] Choose a value based on a condition
!FindInMap Look a value up in Mappings
!ImportValue Name Read an output exported by another stack

Pseudo parameters are values CloudFormation provides: AWS::Region, AWS::AccountId, AWS::StackName and a few more. They let a template avoid hard-coded account IDs and Regions, which makes it portable.

What !Ref returns depends on the resource type, so check the resource's reference page: for a bucket it is the name, for a topic it is the ARN, for a security group it is the ID.

A template resolver

This stand-in reads a template as plain Python data (the JSON form, where !Ref X is written {"Ref": "X"}). It evaluates conditions, orders the resources by their dependencies and resolves Ref, Fn::GetAtt, Fn::Sub, Fn::Join and Fn::If. Physical names that CloudFormation would generate randomly are derived from a hash here so the output is repeatable. It implements only three resource types and a handful of attributes.

Python Resolve Ref, Sub and GetAtt in a template

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the creation order in each run: LogBucket is created before AppBucket because the second one references it, and ReadPolicy comes after AppBucket because ${AppBucket.Arn} inside Fn::Sub is also a dependency. The prod run creates one more resource, the topic, because the condition is true. The same template produced two different environments just by changing one parameter.

Stacks and their lifecycle

You give a template (and parameter values) to CloudFormation and get a stack. With the CLI (read-only here):

Terminal
aws cloudformation deploy \
  --template-file site.yaml \
  --stack-name site-dev \
  --parameter-overrides Environment=dev

aws cloudformation describe-stack-events --stack-name site-dev
aws cloudformation delete-stack --stack-name site-dev

Things to know about how stacks behave:

  • Everything or nothing. If one resource fails to create, CloudFormation rolls back and deletes what it created in this operation, so you do not end up with half a stack. The stack events tell you which resource failed and why. A failed first creation leaves the stack in ROLLBACK_COMPLETE: delete it and try again.
  • Updates are in place when possible. Changing a property may update the resource without interruption, need a restart (some interruption) or force a replacement: a new resource is created and the old one deleted. Replacement of a database or a named bucket can destroy data or fail on a name clash, so know which properties cause it.
  • Deleting a stack deletes its resources. DeletionPolicy: Retain keeps a resource, and Snapshot takes a final snapshot first where the resource type supports it. UpdateReplacePolicy does the same for resources removed by a replacement. Use them for data stores.
  • Let CloudFormation name things. If you omit a name such as BucketName, it generates a unique one, which lets you run several copies of a stack and avoids replacement problems.
  • Stack policies and termination protection guard against accidental updates and deletion of important stacks.

Change sets: preview before you apply

A change set is a dry-run diff. You submit a new version of the template and CloudFormation lists what it would add, modify or remove, and whether each modification needs replacement, without touching anything. You then execute it or discard it.

Terminal
aws cloudformation create-change-set \
  --stack-name site-dev \
  --change-set-name add-log-retention \
  --template-body file://site.yaml

aws cloudformation describe-change-set \
  --stack-name site-dev \
  --change-set-name add-log-retention

aws cloudformation execute-change-set \
  --stack-name site-dev \
  --change-set-name add-log-retention

Reading change sets is the single most useful habit with CloudFormation, because the word Replacement beside a database is a signal to stop.

Drift detection

Someone will eventually change a resource by hand in the console. The stack still thinks the old values are correct. Drift detection compares the live resources with the template and reports resources that were modified or deleted outside CloudFormation. It reports; it does not fix anything. You fix drift by changing the resource back, or by updating the template to match what you want and deploying it.

A change-set and drift stand-in

This stand-in compares two versions of a resources map and prints a change-set-like summary. A small table says which property changes need replacement (modeled on the real behavior of these properties). It then compares the template with "actual" settings to report drift.

Python Preview a change set and detect drift

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The change set shows why previews matter. Renaming the bucket is a replacement, so the old bucket is deleted (data included, unless you retain it) and a new empty one appears. Raising MemorySize and the runtime is a plain in-place update. The new instance is an addition, and the old topic is removed. The drift check finds that someone changed the function's memory to 256 in the console: the template still says 128. Real drift detection reports the same kind of difference for supported resource types.

Organizing big templates

As a template grows, you split it up:

  • Nested stacks use a resource of type AWS::CloudFormation::Stack to include another template, which becomes a child stack. The parent passes parameters in and reads outputs back. Good for reusable building blocks like "a VPC".
  • Cross-stack references use Export in one stack's outputs and Fn::ImportValue in another. Convenient, but an exported value cannot be changed or deleted while another stack imports it, so exports create tight coupling.
  • StackSets deploy one template to many accounts and Regions, which pairs naturally with multi-account governance.
  • Modules, macros and the Transform section (for example AWS SAM) reduce repetition.

Hand-writing YAML for a large system gets tedious. That is the gap the next tools fill: CDK generates CloudFormation templates from real code.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In CloudFormation, what is a stack?

    A stack is the running instance of a template. All its resources share one lifecycle, which is how CloudFormation can roll back or delete them together.
  2. 2. A template has a parameter Environment and a condition IsProd that compares it with prod. What does a resource with Condition IsProd do when Environment is dev?

    A resource whose condition evaluates to false is simply left out of the stack.
  3. 3. Why should you read a change set before executing it?

    A change set is a preview. A modification marked as replacement means the resource is deleted and recreated, which can lose data.
  4. 4. Someone changes a security group in the console. What does drift detection do?

    Drift detection only reports differences. You then decide whether to revert the resource or update the template.
  5. 5. What does DeletionPolicy Retain do?

    With Retain, deleting the stack leaves that resource in place. It is commonly used for buckets and databases that hold data.

Summary

  • A CloudFormation template (Parameters, Conditions, Resources, Outputs and more) describes infrastructure, and a stack is its live instance that is created, updated and deleted as a unit, with rollback on failure.
  • Intrinsic functions such as Ref, GetAtt, Sub, Join and If, plus pseudo parameters, make templates dynamic and portable, and references define the creation order.
  • Some property changes need replacement. Preview them with change sets and protect data with DeletionPolicy and UpdateReplacePolicy.
  • Drift detection reports manual changes but does not fix them.
  • Nested stacks, exports, StackSets and macros help organize large systems.

Next: AWS CDK