Skip to content
Tutorials Center
My learning

Topics

Lesson 23 of 43 ยท Aggregations and Analytics

Aggregations: Metrics and Buckets

Learn Elasticsearch aggregations: avg, sum, stats and cardinality metrics, terms, histogram, date_histogram, range and filters buckets, and why counts can be approximate.

Search answers "which documents match?". Aggregations answer "what does the set of matching documents look like?". How many orders per country? What is the average price? How many sales per month? You get these numbers in the same request as your hits, computed by the cluster instead of by your application.

An aggregation is one of two kinds. A metric aggregation reduces many values to a number or a small group of numbers (average, sum, minimum). A bucket aggregation sorts documents into groups, and every group is called a bucket. Buckets carry a doc_count, and as the next lesson shows (Sub-Aggregations and Pipeline Aggregations) they can contain more aggregations. If you know SQL, metrics are AVG() or SUM(), and buckets are GROUP BY.

The real Elasticsearch cluster cannot run on this site, so the requests below are read-only. Every lesson in this chapter also has runnable Python stand-ins. They compute the same results with plain loops, so you see what the engine does with your data. They are not Elasticsearch.

Two families of aggregations

Metrics Turn many values into one number.
Buckets Sort documents into groups.

Buckets can hold metrics, so you can ask "average price per country".

1 / 2

The request shape

Aggregations live under the aggs key (the long form aggregations also works) next to query. Each aggregation gets a name you choose, then its type, then its settings.

JSON
GET /orders/_search
{
  "size": 0,
  "query": { "range": { "day": { "gte": "2026-01-01" } } },
  "aggs": {
    "avg_price": { "avg": { "field": "price" } },
    "by_country": { "terms": { "field": "country" } }
  }
}

Two details matter. First, "size": 0 tells Elasticsearch you do not need any hits, only the aggregation results, which saves work. Second, aggregations run over the documents selected by the query. Change the query and the numbers change, so a query is also the way to restrict an aggregation to "last month" or "only paid orders".

Aggregations read doc values, a column-style structure stored next to the inverted index (see The Inverted Index). That is why you aggregate on keyword, numeric and date fields. A text field is analyzed into tokens and has no doc values by default, so aggregating on it fails, or you would have to use its .keyword sub-field.

Metric aggregations

The single-value metrics are avg, sum, min and max. Each returns an object like { "value": 27.9 }. A few more are worth knowing.

  • stats returns count, min, max, avg and sum in one go, which is cheaper than four separate aggregations.
  • extended_stats adds variance and standard deviation.
  • value_count counts how many values the field has (a document with an array of three values counts three times).
  • cardinality estimates the number of distinct values.
  • percentiles estimates values like the 95th percentile of a response time.

cardinality and percentiles are approximate on purpose. Counting distinct values exactly would need memory proportional to the number of values, so Elasticsearch uses the HyperLogLog++ algorithm: a small, fixed-size sketch that is accurate to within a small percent. Below a configurable precision_threshold the count is close to exact.

Python Metrics over a list of orders

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The real engine returns the same shapes: a name, then an object with value (or the stats fields). The stand-in's len(set) is an exact distinct count, whereas the real cardinality would use a sketch. On twelve documents the real one would also be exact in practice, because the count stays far below the default precision_threshold.

Bucket aggregations

A bucket aggregation decides, for every document, which bucket or buckets it belongs to.

terms

terms makes one bucket per distinct value of a field, ordered by doc_count descending by default. It is the workhorse behind "top categories" and "orders per country".

JSON
"aggs": {
  "by_category": { "terms": { "field": "category", "size": 5 } }
}

histogram and date_histogram

histogram groups a numeric field into fixed-width intervals. With "interval": 20, a price of 59.99 falls into the bucket with key 40. date_histogram does the same for dates, using calendar units such as month (calendar_interval) or fixed durations such as 7d (fixed_interval). Calendar intervals respect that months have different lengths. Setting "min_doc_count": 0 makes empty buckets appear, so a chart has no gaps.

range and filters

range defines your own buckets: "under 20", "20 to 50", "50 and up". The lower bound is inclusive and the upper bound exclusive. filters creates a bucket per query you name. It is the right tool when buckets are not value ranges, for example "books from Germany" and "games from France". A document can fall in several filters buckets, but in only one range bucket per non-overlapping set of ranges.

Python Buckets: terms, histogram, date_histogram, range, filters

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The totals of the terms, histogram, date_histogram and range outputs are all 12, because each document lands in exactly one bucket. The filters counts do not have to add up to anything, since each filter is independent.

More aggregations worth knowing

Beyond terms and date_histogram, a handful of aggregations cover most remaining needs. All of them go inside "aggs" like the ones above.

date_range, filter and missing

date_range is range for dates and understands date math: { "from": "now-7d/d", "to": "now/d" } means "the last seven whole days". filter (singular) narrows one bucket with a query, which is handy next to a sibling aggregation that should see everything. missing makes a bucket of the documents that have no value for a field, so you can count "no category" instead of silently losing those documents:

JSON
"aggs": {
  "last_week": { "date_range": { "field": "day", "ranges": [ { "from": "now-7d/d", "to": "now/d" } ] } },
  "expensive": { "filter": { "range": { "price": { "gte": 50 } } },
                 "aggs": { "avg_price": { "avg": { "field": "price" } } } },
  "no_category": { "missing": { "field": "category" } }
}

Histogram options

min_doc_count: 0 shows empty buckets between filled ones. To also show empty buckets before the first and after the last value, add extended_bounds, for example { "min": 0, "max": 100 }. hard_bounds does the opposite: it cuts off values outside the range. offset shifts bucket edges, for instance to start days at 06:00.

More statistics

extended_stats returns count, min, max, avg, sum, variance, standard deviation and standard-deviation bounds in one call. percentiles answers "what value is below 95% of the requests?". Its mirror image percentile_ranks answers "what share of requests finished within 200 ms?":

JSON
"aggs": {
  "latency_rank": { "percentile_ranks": { "field": "took_ms", "values": [200, 500] } }
}

Both are approximations computed with a compact sketch (TDigest by default), which is why a result may be a hair off on huge data sets.

significant_terms

terms finds what is common. significant_terms finds what is unusually common in a result set compared with the whole index. Searching complaint tickets for "refund" might show the product X200 in 4% of the hits against 0.3% overall. That product is "significant", even if a bigger product has more raw hits. It is a good way to find recommendations, anomalies or good search suggestions. It needs a decent number of documents to be meaningful, and it works on keyword-like fields.

Geo aggregations

geo_bounds returns the rectangle that contains all matching points, so a map can zoom to fit. geohash_grid (and geotile_grid) bucket points into map cells with a count, which is the data behind a heat map. geo_centroid returns the average position. See Geo and Time-Series Search for the field type that these need.

The stand-in below computes filter, missing, percentile_ranks, extended_stats and a histogram with extended_bounds on a small data set.

JS filter, missing, percentile_ranks, extended_stats, extended_bounds

Press Run (or Ctrl + Enter) to see the console output here.

No output.

How big is a terms result, and how right?

By default terms returns the top 10 buckets, and you raise or lower that with size. The response also reports what was left out:

JSON
"by_country": {
  "doc_count_error_upper_bound": 0,
  "sum_other_doc_count": 140,
  "buckets": [ ... ]
}

sum_other_doc_count is the number of documents that fell in buckets that were not returned. doc_count_error_upper_bound is the worst-case error in the returned counts. It can be nonzero because every shard sends only its own top shard_size terms (default size * 1.5 + 10) to the coordinating node. A term that is big overall but never in a shard's top list can be undercounted or missed. The coordinator sums, for each shard, the count of the last term that shard returned, and that sum is the error bound. If a shard never has to cut anything off, the bound stays 0.

Python Why terms counts can be approximate

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the two lists. The ranking happens to come out right (green, then pink), but the counts are too low: green really has 113 documents and the engine reports 83, because shard 1 cut green off before sending it. pink has 80 but shows 75. The error bound (40 + 35 + 38 = 113 here) honestly warns that counts, and even the order, could be off by that much. Two remedies exist: raise shard_size so shards send more terms, or raise size. Because a single-shard index never merges, its counts are exact. Real data is rarely this adversarial, and results are usually exact when the top terms are clearly ahead.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why do most examples use "size": 0 next to aggs?

    size controls how many hits come back. Aggregations are computed over all matching documents regardless, so setting it to 0 just avoids returning documents you do not need.
  2. 2. Which aggregation estimates the number of distinct values using a small fixed-size sketch?

    cardinality uses HyperLogLog++ and returns an approximate distinct count. value_count counts values, not distinct ones.
  3. 3. A price of 59.99 goes into which bucket of a histogram with interval 20?

    The bucket key is the interval's lower bound: floor(59.99 / 20) * 20 = 40, so the bucket covers 40 up to, but not including, 60.
  4. 4. What does doc_count_error_upper_bound tell you in a terms response?

    Each shard only sends its own top terms, so a term can be undercounted. The upper bound is the sum of the counts of the last term each shard returned.

Summary

  • Metrics compute numbers (avg, sum, min, max, stats, cardinality); buckets create groups with a doc_count.
  • Aggregations run on the documents a query selects and read doc values, so use keyword, numeric and date fields.
  • terms, histogram, date_histogram, range and filters cover most grouping needs.
  • cardinality and percentiles are approximate by design, and terms can be approximate on multi-shard indexes.
  • Check doc_count_error_upper_bound and sum_other_doc_count, and raise size or shard_size when accuracy matters.

Next: put aggregations inside aggregations in Sub-Aggregations and Pipeline Aggregations.