Skip to content
Tutorials Center
My learning

Topics

Lesson 2 of 41 ยท Getting Started

Installing dbt and Setting Up a Project

Install dbt Core with pip, choose an adapter, run dbt init, and understand profiles.yml targets and connection settings, with a runnable stand-in for target selection.

Setting up dbt takes three steps: install dbt Core together with an adapter for your warehouse, create a project, and tell dbt how to connect through a file called profiles.yml. This lesson walks through each step. The shell commands and YAML here are read-only because they act on your machine and a real warehouse, but we finish with a runnable stand-in that shows exactly how dbt picks a connection.

Installing dbt Core

dbt Core is a Python package. The recommended way is a virtual environment so dbt's dependencies stay separate from the rest of your system.

Terminal
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
python -m pip install dbt-core dbt-duckdb
dbt --version

You never install dbt-core alone. It contains the engine, but each warehouse needs its own adapter package that knows how to talk to it:

Warehouse Adapter package
PostgreSQL dbt-postgres
Snowflake dbt-snowflake
BigQuery dbt-bigquery
Databricks dbt-databricks
Redshift dbt-redshift
DuckDB (local file, great for learning) dbt-duckdb
SQL Server dbt-sqlserver

For learning on your own computer, DuckDB is the easiest choice: it is a single file, needs no server and is fast. Installing the adapter also installs a compatible dbt-core, and dbt --version lists both.

Pin versions in a requirements.txt (for example dbt-core==1.10.*). A team where everyone has a different dbt version gets different results and confusing errors.

Creating a project with dbt init

dbt init scaffolds a project and, the first time, asks you a few questions to write your profile.

Terminal
dbt init shop_analytics
cd shop_analytics

The result is a folder with this shape (we study each part in the next chapter):

TEXT
shop_analytics/
โ”œโ”€โ”€ dbt_project.yml
โ”œโ”€โ”€ models/
โ”‚   โ””โ”€โ”€ example/
โ”œโ”€โ”€ seeds/
โ”œโ”€โ”€ snapshots/
โ”œโ”€โ”€ macros/
โ”œโ”€โ”€ tests/
โ””โ”€โ”€ analyses/

dbt_project.yml is the project's control file. It holds the project name, the profile it should use and where things live. The profile name is the link to your connection settings:

YAML
name: "shop_analytics"
version: "1.0.0"
profile: "shop_analytics"

model-paths: ["models"]
seed-paths: ["seeds"]
test-paths: ["tests"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

models:
  shop_analytics:
    +materialized: view

profiles.yml: how dbt connects

Connection details are not stored in the project, because they contain secrets and differ per person. They live in profiles.yml, by default in ~/.dbt/ (your home directory), or in the project folder if you point dbt there with --profiles-dir ..

YAML
shop_analytics:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: dev.duckdb
      threads: 4
    prod:
      type: postgres
      host: warehouse.example.com
      port: 5432
      user: "{{ env_var('DBT_USER') }}"
      password: "{{ env_var('DBT_PASSWORD') }}"
      dbname: analytics
      schema: analytics
      threads: 8

The pieces:

  • The top-level key (shop_analytics) must equal profile: in dbt_project.yml.
  • outputs lists named targets, one per environment. Here dev is a local DuckDB file and prod is a Postgres server.
  • target: dev is the default. Override it for one command with dbt run --target prod.
  • threads is how many models may build in parallel.
  • Secrets come from environment variables through env_var(), so the file never holds a password in plain text and is safe to share the structure of.

Connection fields differ by adapter (Snowflake wants an account, BigQuery a project and dataset), so check the adapter's page. Common to all are type, threads, and a schema (or dataset) where dbt creates objects.

Checking the connection

Terminal
dbt debug

dbt debug checks that dbt_project.yml is valid, finds your profile, tests the connection and prints what it found. When something is wrong, this is the first command to run.

Stand-in: how dbt chooses a target

Here is a small Python program, not dbt, that reads a profile in the same format and picks a target the way dbt does. It uses real YAML and real Jinja, so you can see how env_var() works.

Python Stand-in: reading profiles.yml

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The same file served two connections, and the secret came from the environment rather than the file. If you remove DBT_PASSWORD from fake_env and run it again, you get dbt's real error message for a missing variable.

Python Stand-in: a missing environment variable

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Common setup mistakes

  • Profile name mismatch. dbt_project.yml says profile: shop_analytics but profiles.yml has no such key. dbt reports "Could not find profile named ...".
  • Wrong adapter. The type: in a target must match an installed adapter.
  • Committing credentials. Keep profiles.yml out of Git, or use env_var() for every secret.
  • Mixing up dev and prod. Make dev the default target so a careless dbt run never touches production.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do you install an adapter such as dbt-postgres?

    dbt-core is the engine, and each warehouse needs its own adapter that knows its SQL dialect and connection method.
  2. 2. Where does dbt look for connection credentials by default?

    Connections live in profiles.yml, which defaults to the .dbt folder in your home directory, so secrets stay out of the project.
  3. 3. How do you run a command against the prod target for one run only?

    --target selects one of the named outputs from the profile without changing the default.
  4. 4. Which command checks that your project and connection are set up correctly?

    dbt debug validates the project file, locates the profile and tests the warehouse connection.

Summary

  • Install dbt-core together with one adapter in a virtual environment; DuckDB is ideal for practice.
  • dbt init creates a project; dbt_project.yml names it and links it to a profile.
  • profiles.yml holds named targets (dev, prod), threads and credentials, ideally through env_var().
  • dbt debug verifies the setup; --target switches environments.

Next: Your First dbt Run