Skip to content
Tutorials Center
My learning

Topics

Lesson 14 of 37 ยท Hosts and Items

Zabbix History, Trends and Housekeeping: Storage Periods and Database Size

Understand how Zabbix stores raw history and hourly trends, how the housekeeper deletes old data, how to estimate database size, and how partitioning and TimescaleDB help.

Every value Zabbix collects goes into the database, and the database grows forever unless you set retention rules. Zabbix keeps two kinds of numeric data: history (every raw value) and trends (one aggregate per item per hour). Choosing the right retention for each is the difference between a fast, small database and a slow, huge one. This lesson explains both, how the housekeeper cleans up, and how to estimate size before you collect.

Zabbix does not run on this site. The stand-in code does the calculations you would do on a spreadsheet.

History

History is the raw stream of values exactly as stored after preprocessing: one row per collected value. History tables exist per type: history (float), history_uint (unsigned), history_str, history_log, history_text, history_bin. History is what Latest data, graphs of recent time and triggers read. It is the biggest consumer of space.

In the item form the History storage period field says how long to keep these values. A common default is 31d (31 days); 0 means "do not store history" (triggers still run). Triggers that look back, such as avg(/host/key,1h), only need recent history, so there is no need to keep raw data for months just for alerts.

For numeric items, Zabbix also calculates trends: for every hour it stores min, avg, max and the number of values (num). Trends use roughly 1/60th of the space (at 1 value per minute) and still draw useful long-range graphs, since a month of hourly min/avg/max shows spikes and growth well. The Trend storage period (default 365d) says how long to keep them. Text, character and log items have no trends.

A graph automatically switches from history to trends when the displayed range is long enough that history would not cover it or that one pixel represents more than a few values.

Retention strategy

Data Raw history Trends
Critical metrics (CPU, memory, traffic) 7 to 31 days 1 to 2 years
Fast checks (ping every 10 s) 3 to 7 days 1 year or off
Slow checks (disk space) 31 days 2 years
Text/log data as short as possible n/a
Debug items 1 to 2 days 0

Because trends depend on the numeric history, you cannot keep trends without writing history in the first place, even if you keep history for only a day.

Estimating the database size

Use these planning numbers (for a typical SQL database, with indexes):

  • History about 90 bytes per value (numeric).
  • Trends about 90 bytes per item per hour.
  • Events and other tables are small compared with these.
Python Stand-in: database size estimate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The result shows why a short history period matters: the 31 days of raw history dwarf a whole year of trends. Cutting history from 31 days to 7 shrinks that part by 77%.

Python Stand-in: effect of changing retention

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The housekeeper

The housekeeper is a server process that periodically deletes outdated data. Its settings are in Administration > Housekeeping:

  • Enable internal housekeeping for events and alerts, IT services, user sessions, audit log, history and trends. Each area has its own retention (for example events 365 days, audit log 31 days).
  • Override item history period and Override item trend period: if on, the global values replace the per-item storage periods (useful as a global cap).
  • The server config option HousekeepingFrequency (default 1 hour) and MaxHousekeeperDelete (default 5000 rows per task per run) control how often and how many rows the server deletes.

On a big database, deletes are slow and can cause lag. Signs: the "housekeeper is busy" internal item stays high, and the database CPU is high with many DELETE statements. The cure is table partitioning (or TimescaleDB), described next.

Partitioning and TimescaleDB

Deleting rows one by one is expensive; dropping a whole partition is instant. With partitioning, the history* and trends* tables are split by time (daily for history, monthly for trends). A scheduled job creates future partitions and drops old ones, and you disable housekeeping for history and trends in the frontend (set the "Enable internal housekeeping" options for history and trends off).

TimescaleDB (a PostgreSQL extension) does this natively: hypertables with compression and chunk dropping. Zabbix supports it directly: enable it in the frontend (Administration > Housekeeping, "Override item history period" and "Enable compression"). It is the recommended path for large PostgreSQL installations; compression can reduce history size by up to 90%.

Python Stand-in: delete rows vs drop partitions

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Monitoring the monitor

Zabbix exposes internal items you should watch:

  • zabbix[wcache,history,pfree]: free space in the history write cache (should stay high).
  • zabbix[process,history syncer,avg,busy]: how busy the DB writers are.
  • zabbix[process,housekeeper,avg,busy]: housekeeper activity.
  • zabbix[queue]: items waiting beyond their interval.

The template Zabbix server health contains these and triggers for them. A full history cache means the database cannot keep up with the NVPS: add faster storage, tune the database (innodb buffer pool, innodb_flush_log_at_trx_commit), or reduce the load.

Practical tips

  1. Set history to what triggers and troubleshooting actually need (days to a month), and keep long-range view with trends.
  2. Turn off history for items that are only used for triggers on last(), but only if you never need to see them in a graph.
  3. Use Discard unchanged for slow-changing values (preprocessing).
  4. Raise intervals for items that do not need minute resolution.
  5. Plan disk space with margin: databases need space for indexes, temporary tables and backups.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a trend record contain for each hour?

    Trends hold hourly min, avg, max and num for numeric items.
  2. 2. Which data uses the most database space in a typical installation?

    Raw history grows with every collected value.
  3. 3. What is the main benefit of partitioning history tables?

    Dropping a partition is nearly instant compared with deleting millions of rows.
  4. 4. Which item types have no trends?

    Only numeric values can be aggregated into min, avg and max.

Summary

  • History = raw values; trends = hourly min/avg/max for numeric items.
  • Choose history in days and trends in months or years; history dominates size.
  • The housekeeper deletes expired data; at scale use partitioning or TimescaleDB.
  • Estimate size from NVPS x retention x about 90 bytes per value.
  • Monitor Zabbix itself with its internal items and the server health template.

Next: Trigger expressions