Skip to content
Tutorials Center
My learning

Topics

Lesson 13 of 37 ยท DAGs and Operators

Airflow Templating with Jinja: ds, Macros and Rendered Fields

Use Jinja templates in Apache Airflow: {{ ds }}, logical_date, data_interval, params, macros, custom filters, template files and how to debug rendered values.

A task usually needs to know which day it is working on. Hard-coding a date into a DAG would make every run identical, so Airflow lets you write templates: text with placeholders such as {{ ds }} that Airflow fills in just before the task runs. Templating is built on the Jinja library and is one of the first things that makes Airflow DAGs feel truly dynamic.

Rendering happens inside Airflow, so the DAG code is read-only here. The stand-ins use Python's own string and datetime tools to mimic what {{ ds }}, {{ ds_nodash }} and date arithmetic produce, and a small renderer so you can try templates yourself.

The basics

Python
from airflow.providers.standard.operators.bash import BashOperator

export = BashOperator(
    task_id="export",
    bash_command="""
      echo "Run: {{ run_id }}"
      echo "Logical date: {{ ds }}"
      pg_dump mydb > /backups/mydb_{{ ds_nodash }}.sql
    """,
)

Before running, Airflow renders the string: it looks for {{ ... }} expressions and replaces them using the context of the current task instance. For a run with logical date 5 March 2025, the third line becomes /backups/mydb_20250305.sql.

Which fields are templated?

Only fields listed in the operator's template_fields. For BashOperator that includes bash_command, env and cwd; for PythonOperator it includes op_args and op_kwargs. Look at the operator's documentation or the Rendered Template tab in the UI to see what was rendered. A field that is not templated shows {{ ds }} literally.

Common context variables

Variable Example value Meaning
{{ ds }} 2025-03-05 Logical date, YYYY-MM-DD
{{ ds_nodash }} 20250305 Same without dashes
{{ ts }} 2025-03-05T00:00:00+00:00 Logical date and time (ISO)
{{ logical_date }} datetime object The logical date as a datetime
{{ data_interval_start }} / {{ data_interval_end }} datetimes Bounds of the data interval
{{ run_id }} scheduled__2025-03-05T00:00:00+00:00 The DAG run id
{{ dag.dag_id }}, {{ task.task_id }} strings Names
{{ ti.try_number }} 1 Attempt number
{{ params.city }} Oslo A DAG or task param
{{ dag_run.conf }} dictionary JSON passed when triggering
{{ var.value.region }} eu An Airflow Variable (see Variables and Params)
{{ ti.xcom_pull(task_ids="extract") }} any A value another task returned

The exact date variables depend on the run type (see Data intervals); in Airflow 3 some runs have no logical date, so prefer data_interval_start and data_interval_end when a run needs a time range.

Jinja basics

TEXT
{{ expression }}                    print a value
{{ ds | replace("-", "/") }}        filters transform values
{% if params.full %}...{% endif %}  statements (if, for, set)
{# a comment #}

A loop inside a templated SQL string:

Python
sql = """
SELECT *
FROM sales
WHERE sale_date >= '{{ data_interval_start | ds }}'
  AND sale_date <  '{{ data_interval_end | ds }}'
  AND region IN (
    {% for r in params.regions %}'{{ r }}'{{ "," if not loop.last }}{% endfor %}
  )
"""

Airflow adds helpful filters such as ds (format as a date) and ts. Dates are normal Python objects, so {{ logical_date.strftime("%Y/%m/%d") }} and {{ logical_date.year }} also work.

Macros

Airflow exposes a few helpers under macros:

TEXT
{{ macros.ds_add(ds, 7) }}                      date + 7 days     -> 2025-03-12
{{ macros.ds_add(ds, -1) }}                     yesterday
{{ macros.ds_format(ds, "%Y-%m-%d", "%d %b") }} reformat          -> 05 Mar

Plain Python date maths with logical_date + macros.timedelta(days=1) also works, because timedelta and datetime are available in the macros namespace.

Template files

Long SQL belongs in a file, not in a Python string. Give the operator the file name and a search path:

Python
from airflow.sdk import DAG
with DAG(
    dag_id="sql_files",
    start_date=datetime(2025, 1, 1),
    schedule="@daily",
    template_searchpath=["/opt/airflow/include/sql"],
) as dag:
    BashOperator(task_id="x", bash_command="run_report.sh ")   # note the space

For SQL operators, sql="daily_sales.sql" loads and renders the file. Files ending with a templated extension (for example .sql) listed in template_ext are read and rendered automatically.

Templates inside @task functions

In TaskFlow functions you rarely need Jinja: ask for the context directly.

Python
from airflow.sdk import task, get_current_context

@task
def report():
    ctx = get_current_context()
    day = ctx["ds"]
    print(f"Building report for {day}")

You can also name context keys as parameters: def report(ds=None, params=None): ... and Airflow fills them in.

Debugging templates

  1. Open the task instance in the UI and choose Rendered Template.
  2. From the CLI: airflow tasks render <dag_id> <task_id> <logical_date>.
  3. Remember that rendering happens at run time, not parse time. Never try to use Jinja results to decide how many tasks to create; use dynamic task mapping for that.
  4. If a string contains literal {{, such as a JSON template for another system, escape it with {% raw %} ... {% endraw %}.

Stand-in: what the date templates produce

ds, ds_nodash, ts and macros.ds_add are all simple date formatting. This stand-in shows the output for a given logical date.

Python Stand-in: ds, ds_nodash, ts and ds_add

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Stand-in: a tiny template renderer

This renderer replaces {{ name }} and dotted names like params.city from a context dictionary. It lacks Jinja's loops and filters, but shows the principle.

Python Stand-in: render {{ placeholders }}

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

(Real Jinja renders an undefined name as an empty string by default. Airflow can be set to raise instead, with template_undefined/jinja_environment_kwargs, which is what this stand-in mimics.)

Common mistakes

  • Templating a field that is not in template_fields, then seeing literal braces.
  • Using datetime.now() in a command instead of {{ ds }}, which breaks re-runs and backfills because "now" is not the run's date.
  • Forgetting that the string is rendered only at run time; syntax errors appear when the task starts.
  • Missing the trailing space after a .sh path in bash_command.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. When does Airflow render a template like {{ ds }}?

    Rendering is per task instance at run time, with that run's context.
  2. 2. Why use {{ ds }} instead of datetime.now() in a task?

    A re-run next week must still process the original day, which only the run's own date guarantees.
  3. 3. A templated command shows {{ ds }} literally in the Rendered Template tab. What is the likely cause?

    Only template_fields are rendered.
  4. 4. What does {{ macros.ds_add(ds, -1) }} give for ds = 2025-03-05?

    ds_add adds the given number of days; -1 means the day before.

Summary

  • Templates use Jinja: {{ ds }}, filters, {% if %}/{% for %}, macros and dotted access to params.
  • Only operator fields listed in template_fields are rendered, at run time.
  • Use template variables rather than "now" so re-runs and backfills are reproducible.
  • Check the Rendered Template tab or airflow tasks render when something looks wrong.

Next: Schedules: cron, presets and timedelta