Skip to content
Tutorials Center
My learning

Topics

Lesson 18 of 37 ยท Variables, Outputs and Data Sources

Data Sources: Reading Existing Infrastructure

Use Terraform data sources to read existing infrastructure: aws_ami, availability zones, caller identity, IAM policy documents, filters, timing, errors and when to prefer resources.

A resource says "make this thing exist and keep it that way." A data source says "look up something that already exists and tell me about it." Data sources let your configuration use information from outside its own control: the latest Ubuntu image, the availability zones in a region, the account ID you are running in, a VPC your network team created, or a JSON policy document built from HCL. They never create, change or delete anything.

The data blocks are read-only here; the stand-ins imitate filtering, "most recent" selection and the common failure modes.

The data block

HCL
data "aws_ami" "ubuntu" {
  most_recent = true
  owners      = ["099720109477"]   # Canonical

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-amd64-server-*"]
  }
}

resource "aws_instance" "web" {
  ami           = data.aws_ami.ubuntu.id
  instance_type = "t3.micro"
}

Like a resource, a data block has a type (aws_ami) and a local name (ubuntu). You read its attributes through data.<type>.<name>.<attribute>. The arguments inside are query arguments: they describe what to find. The provider documentation has a separate "Data Sources" section listing which arguments narrow the search and which attributes come back.

When data is read

Terraform reads data sources during plan (it refreshes them every time) whenever all their arguments are known. If an argument depends on a value only known after apply, such as the ID of a resource that does not exist yet, the read is deferred to apply, and its results are (known after apply) in the plan. A depends_on on a data source also defers it. Keep data source queries based on variables, constants and long-lived infrastructure to keep plans informative.

flowchart LR
    V["variables, constants"] --> D["data source<br/>(read at plan)"]
    D --> R["resource<br/>uses data.x.y.id"]
    N["new resource<br/>(known after apply)"] -.-> D2["data source<br/>(read at apply)"]

A data source whose inputs are known is read at plan time; one that depends on a resource not yet created is read during apply.

Everyday data sources

Data source What it returns
aws_caller_identity The account ID, ARN and user of the credentials in use
aws_region The current region name
aws_availability_zones Zones available in the region
aws_ami An image matching filters (newest, from a trusted owner)
aws_vpc, aws_subnets Existing network objects found by tags or IDs
aws_iam_policy_document An IAM policy built from HCL, with a json attribute
aws_secretsmanager_secret_version A stored secret (the value goes to state)
google_compute_image, azurerm_resource_group Equivalents on other clouds
http, external Call a URL or a script and use the result
terraform_remote_state Outputs of another configuration's state
cloudflare_zone, github_repository Look up SaaS objects by name

Using the account ID and zones is typical:

HCL
data "aws_caller_identity" "current" {}
data "aws_availability_zones" "available" {
  state = "available"
}

locals {
  account_id = data.aws_caller_identity.current.account_id
  azs        = slice(data.aws_availability_zones.available.names, 0, 3)
}

Building policies with a data source

aws_iam_policy_document is a pure computation, no API call, and a favorite for readable policies:

HCL
data "aws_iam_policy_document" "read_logs" {
  statement {
    effect    = "Allow"
    actions   = ["s3:GetObject", "s3:ListBucket"]
    resources = [aws_s3_bucket.logs.arn, "${aws_s3_bucket.logs.arn}/*"]
  }
}

resource "aws_iam_policy" "read_logs" {
  name   = "read-logs"
  policy = data.aws_iam_policy_document.read_logs.json
}

You get validation, merging (source_policy_documents) and interpolation without writing raw JSON strings.

Python A data source filters and picks the newest match (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The last line shows the security lesson: omit or loosen owners and most_recent could select an image someone else published. Always restrict AMI lookups to a trusted owner.

Common errors

  • "Your query returned no results": a filter that matches nothing, a wrong region or a tag typo.
  • "returned more than one result": add filters or most_recent = true where the data source allows it.
  • Plans differ run to run: most_recent finds a newer image tomorrow, so the instance is planned for replacement. If you want stability, pin the ID in a variable and update it deliberately.
  • Data source reads a resource that is being replaced: the read may happen late; avoid depends_on on data sources.

Reading an existing VPC

HCL
data "aws_vpc" "shared" {
  tags = {
    Name = "shared-network"
  }
}

data "aws_subnets" "private" {
  filter {
    name   = "vpc-id"
    values = [data.aws_vpc.shared.id]
  }
  tags = {
    Tier = "private"
  }
}

resource "aws_instance" "app" {
  for_each  = toset(data.aws_subnets.private.ids)
  subnet_id = each.value
  ami       = data.aws_ami.ubuntu.id
}

Notice the risk: for_each keys here come from a data source. If the subnets exist already, the IDs are known at plan time and it works. If the VPC is created in the same configuration, the IDs are unknown and Terraform errors. Data sources are for objects that exist before the apply.

Sharing data with other configurations

terraform_remote_state reads the outputs of another configuration:

HCL
data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "example-tfstate"
    key    = "network/terraform.tfstate"
    region = "eu-central-1"
  }
}

locals {
  vpc_id = data.terraform_remote_state.network.outputs.vpc_id
}

It couples this configuration to the other one's backend and requires read access to the whole state, which may contain secrets. Many teams instead read the resources directly by tag with ordinary data sources, or publish values to a parameter store.

Data sources versus resources versus variables

You want to... Use
Create and manage an object resource
Read something someone else manages data
Receive a value from the caller variable
Adopt an existing object into Terraform's management import block plus a resource

Rule of thumb: if Terraform should own the lifecycle, use a resource; if it merely needs to know about the thing, use a data source.

Resource or data?

Resource Terraform creates and owns it.
Data source Terraform only reads it.
Variable The caller provides it.
Import Adopt an existing object.
1 / 2

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a data block do?

    Data sources are read-only lookups. They never create, modify or delete infrastructure.
  2. 2. How do you read the id of data "aws_ami" "ubuntu"?

    References to data sources start with data., then type, name and attribute.
  3. 3. Why is a data source with depends_on on an unapplied resource problematic for plans?

    A deferred data source returns (known after apply) values, which can hide changes and break for_each keys.
  4. 4. Why restrict owners in an aws_ami lookup with most_recent = true?

    most_recent picks the newest match among everything that passes the filters, including images from untrusted publishers.

Summary

  • A data block reads existing objects; reference it as data.type.name.attribute.
  • Data sources are refreshed on every plan, and deferred to apply when their inputs are unknown.
  • Common ones: caller identity, region, availability zones, AMIs, VPCs and subnets, policy documents, http, external, terraform_remote_state.
  • Zero or multiple matches are errors; make filters precise and restrict trusted owners.
  • Use resources for things Terraform owns, data sources for things it only needs to know.

Next: Terraform State: What It Is and Why It Exists