Skip to content
Tutorials Center
My learning

Topics

Lesson 8 of 37 ยท Agent and Integrations

Datadog Integrations and Checks: conf.d, nginx, PostgreSQL and More

Turn on Datadog integrations: conf.d layout, conf.yaml and conf.yaml.example, init_config and instances, service checks, nginx and PostgreSQL examples, and testing a check.

CPU and memory come for free, but the real value of Datadog appears when it understands your software. An integration is a packaged connection to one technology (nginx, PostgreSQL, Redis, Kafka, a cloud service) that collects the right metrics, ships a ready-made dashboard and often log parsing rules. Over 900 integrations exist, so most of your stack is already covered.

You configure integrations on the host, in files. Datadog does not run on this site, so the configs below are read-only examples and the stand-ins test your understanding of how the files fit together.

Three kinds of integration

  1. Agent-based: a check runs inside the Agent on your host and connects to local or remote software (nginx, MySQL, Redis).
  2. Crawler-based (cloud and SaaS): Datadog calls the provider's API with credentials you give it, with no Agent. AWS, Azure, GCP, GitHub and Slack work this way. See cloud integrations.
  3. Library-based: instrumentation libraries inside your application (APM tracing, Lambda).

The conf.d layout

Each Agent-based integration has its own folder under conf.d:

TEXT
/etc/datadog-agent/conf.d/
  nginx.d/
    conf.yaml.example      # all options, documented; the Agent ignores it
    conf.yaml              # your config: the Agent loads this
  postgres.d/
    conf.yaml

The folder is named <integration>.d. The Agent only loads files ending in .yaml (not .example). Copy the example, edit it, and restart the Agent. Newer setups also use auto_conf.yaml for autodiscovery (see the container lessons).

Anatomy of a check configuration

YAML
init_config:
  # settings shared by all instances of this check (often empty)

instances:
  - nginx_status_url: http://localhost:81/nginx_status/
    min_collection_interval: 30
    tags:
      - service:storefront
      - env:prod
  • init_config holds settings for the whole check.
  • instances is a list; each item is one thing to monitor. Two nginx servers means two instances.
  • min_collection_interval (seconds, default 15) lowers how often this instance runs.
  • tags are added to every metric from this instance.

Example: nginx

nginx needs the stub_status module exposed on a local-only address:

NGINX
server {
  listen 127.0.0.1:81;
  location /nginx_status {
    stub_status;
    allow 127.0.0.1;
    deny all;
  }
}

Then nginx.d/conf.yaml as above. After a restart, metrics such as nginx.net.connections, nginx.net.request_per_s and the service check nginx.can_connect appear.

Example: PostgreSQL

Create a read-only monitoring user first:

SQL
CREATE USER datadog WITH PASSWORD '<PASSWORD>';
GRANT pg_monitor TO datadog;
GRANT SELECT ON pg_stat_database TO datadog;
YAML
init_config:
instances:
  - host: localhost
    port: 5432
    username: datadog
    password: "ENC[pg_password]"
    tags:
      - env:prod

Metrics include postgresql.connections, postgresql.rows_returned and postgresql.database_size. Give monitoring users the least privilege, and use secrets management for the password (see datadog.yaml configuration).

Service checks

Besides metrics, a check can report a service check: a status of OK, WARNING, CRITICAL or UNKNOWN. nginx.can_connect is CRITICAL if the Agent cannot reach the status URL. You build monitors on service checks to alert when something is down.

Testing and debugging a check

Terminal
sudo -u dd-agent datadog-agent check nginx       # run once and print results
sudo datadog-agent configcheck                   # shows loaded configs
sudo datadog-agent status                        # per-check counts and errors

check nginx runs the check one time and prints every metric, so you see exactly what would be sent. Add --log-level debug for details. In status, the Checks section lists each check with Total Runs, metric count, and the last error message. Typical failures: a closed port, wrong credentials, or a file the dd-agent user cannot read.

Where to find an integration

Integrations > Integrations lists everything; each tile has an Install or Configure tab with the exact YAML and, once data flows, installs a dashboard and monitor templates automatically. Some integrations ship as separate packages: sudo -u dd-agent datadog-agent integration install datadog-<name>==<version> pins a specific version.

Python Stand-in: load a check config and expand instances

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Python Stand-in: a service check status from a connection result

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Habits

  • Start with the integration's tile: copy its recommended config, not a blog post's.
  • Tag each instance with service and env.
  • Create dedicated read-only database users.
  • Watch the collection time of heavy checks; raise min_collection_interval for slow ones.
  • Check datadog-agent status after every change; do not assume success.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which file does the Agent actually load for the nginx integration?

    The example file is documentation; copy it to conf.yaml to activate the check.
  2. 2. In a check config, what does the instances list represent?

    Each item in instances is one monitored target with its own settings.
  3. 3. Which command runs a check once and prints its metrics?

    check <name> runs the check one time so you see what it would send.
  4. 4. What statuses can a service check have?

    Service checks report one of the four states OK, WARNING, CRITICAL or UNKNOWN.

Summary

  • Integrations come in three kinds: Agent-based, crawler-based (cloud/SaaS) and library-based.
  • Agent integrations live in conf.d/<name>.d/conf.yaml with init_config and instances.
  • Checks emit metrics and service checks; test with datadog-agent check <name>.
  • Use least-privilege monitoring accounts and secrets for credentials.

Next: Writing Custom Agent Checks