Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 41 ยท Getting Started

What Is dbt?

Learn what dbt (data build tool) does, why analytics teams use it, how it fits into ELT, and what you can and cannot run on this site while following the course.

dbt (short for data build tool) is a command-line tool that turns SELECT statements into tables and views in your data warehouse. You write SQL files. dbt wraps each one in the right CREATE VIEW or CREATE TABLE statement, works out the order they must run in, runs them, and can test the results. That is the whole idea, and it is enough to change how a team works with data.

Before dbt, transformation code often lived in scheduled scripts, stored procedures or a BI tool's hidden "calculated fields". Nobody could see how a number was produced, and nothing was tested. dbt moves that logic into plain text files in a Git repository, so it can be reviewed, versioned, tested and documented like any other software.

What runs here. dbt needs a real warehouse (Snowflake, BigQuery, Postgres, DuckDB and so on), and a browser cannot host one. On this site, the SQL you see in a model runs in an in-browser SQLite database with sample shop data, and the Jinja templating that makes dbt powerful runs in the real Jinja2 library inside Python. The dbt commands and YAML files are shown as read-only code, and every stand-in is labelled. The ideas transfer one to one to a real project.

ELT: where dbt fits

Older systems used ETL: extract data from a source, transform it on a separate server, then load the result. Modern cloud warehouses are fast and cheap enough to do the heavy lifting themselves, so teams switched to ELT: load raw data first, then transform it inside the warehouse.

ELT in three steps

Extract Copy data out of apps and databases
Load Land it raw in the warehouse
Transform Clean and model it with SQL

dbt is only the T. Tools such as Fivetran or Airbyte do the E and L.

1 / 2

dbt does not extract or load data. It does not store data either: your warehouse does. dbt is the layer in between that sends SQL to the warehouse and keeps your transformation logic organized.

What a dbt model looks like

A model is one .sql file containing one SELECT. Here is a model that cleans a raw customers table. The raw table lives in our sample warehouse, and this query is exactly what the file models/staging/stg_customers.sql would contain:

SQL A model is just a SELECT ยท database: shop

Press Run (or Ctrl + Enter) to see the result here.

Notice what is missing: there is no CREATE TABLE, no INSERT, no DROP. dbt adds that boilerplate for you, depending on how you configure the model. Change one setting and the same file becomes a view, a table, or an incrementally updated table.

The big ideas

Models reference each other. Instead of hard-coding table names, you write {{ ref('stg_customers') }}. dbt replaces it with the real table name for the current environment and records that one model depends on another. From those references it builds a DAG (directed acyclic graph) and runs models in a valid order.

SQL
select * from {{ ref('stg_customers') }}

Environments are first class. The same project runs against a developer's private schema and against production just by switching a target. Nobody edits table names by hand.

Tests are part of the project. You declare that a column must be unique or never null, and dbt test turns that into SQL that looks for violations. See tests in the testing chapter of this course for the full story.

Documentation is generated. Descriptions you write in YAML become a browsable site with a lineage graph.

Jinja turns SQL into a template. Loops, conditions and reusable macros let you generate repetitive SQL instead of copy-pasting it. You will meet this in the Jinja chapter, and we will use the real Jinja engine to see it working.

A tiny dbt in forty lines

To make the mechanism concrete, here is a toy version of what dbt does with ref(). It is not dbt, only a stand-in written with the real Jinja2 library: it renders each model, replaces ref with a table name, and discovers dependencies while rendering.

Python Stand-in: how ref() builds a graph

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Real dbt does the same three things at a much larger scale: render, collect references, run in dependency order. It also supports parallel threads, so independent models run at the same time.

Who uses dbt, and for what

The people who use dbt most are analytics engineers: they sit between data engineers (who move data) and analysts (who ask questions). Typical dbt projects include:

  • cleaning and renaming raw columns into consistent staging tables;
  • building tables for finance, marketing or product dashboards;
  • calculating metrics (revenue, active users) once, in one place;
  • keeping a history of slowly changing data with snapshots;
  • running data quality checks every morning before anyone opens a dashboard.

dbt Core, dbt Cloud and Fusion

You will see three names. dbt Core is the open-source command-line tool, and it is what most of this course uses. dbt Cloud (dbt platform) is a hosted product that adds a web IDE, scheduling and alerts. dbt Fusion is a newer engine from dbt Labs, written in Rust, that is compatible with most Core projects and adds faster parsing and richer editor support. Everything you write as models, YAML and Jinja is the same project format in all three. A later lesson compares them in detail.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which part of ELT does dbt perform?

    dbt only transforms data that is already in the warehouse. Other tools extract and load it.
  2. 2. What does a dbt model file contain?

    A model is one SELECT. dbt generates the CREATE VIEW or CREATE TABLE around it.
  3. 3. What is the main purpose of ref() inside a model?

    ref() resolves to the right table name for the current environment and lets dbt build the dependency graph.
  4. 4. Where does the data stored by a dbt project actually live?

    dbt stores no data. It sends SQL to the warehouse, which creates and holds the tables.

Summary

  • dbt is the T in ELT: it turns SQL SELECT files into tables and views in your warehouse.
  • Models are plain SQL files; ref() links them and lets dbt build a dependency graph (DAG).
  • Tests, documentation, environments and Jinja templating are built in.
  • dbt Core is open source; dbt Cloud and Fusion are other ways to run the same project format.
  • On this site, SQL stand-ins run in SQLite and Jinja runs for real; dbt commands are read-only.

Next: Installing dbt and Setting Up a Project