Skip to content
Tutorials Center
My learning

Topics

Lesson 12 of 43 ยท Mappings and Data Modeling

Objects, Arrays and Nested Fields

Learn how Elasticsearch flattens objects and arrays of objects, why that causes false matches, and how the nested type and join field fix it.

JSON documents are full of objects and lists, and Elasticsearch handles both without special syntax. But it does not store them the way you might expect. Under the hood a Lucene document is a flat list of fields, so objects are flattened into dotted names, and a list of objects loses the link between the values inside each object. The result is a classic surprise: a query that should match nothing matches a document anyway.

In this lesson you will see the flattening problem, fix it with the nested type, and learn when a join field or plain denormalization is the better choice. The requests are read-only; JavaScript stand-ins reproduce the behavior.

Objects are flattened

Take this document:

JSON
{
  "title": "Trail shoe",
  "maker": { "name": "Acme", "country": "DE" }
}

Elasticsearch indexes it as if it were:

JSON
{ "title": "Trail shoe", "maker.name": "Acme", "maker.country": "DE" }

You query maker.name with a dotted path. The mapping has an object type for maker (you rarely write "type": "object" yourself, since it is the default for a field with properties). Single objects work exactly as you expect.

Arrays of values

Any field can hold a list of values, and a query matches if any value matches:

JSON
{ "title": "Trail shoe", "tags": ["running", "trail", "outdoor"] }

A term query for tags: "trail" matches. All values of the array must have the field's type (you cannot mix numbers and objects in one array).

The array-of-objects pitfall

Now combine the two. A product has a list of reviews, each with an author and a rating:

JSON
{
  "title": "Trail shoe",
  "reviews": [
    { "author": "ana", "rating": 5 },
    { "author": "bo",  "rating": 2 }
  ]
}

Flattening gives:

JSON
{ "reviews.author": ["ana", "bo"], "reviews.rating": [5, 2] }

The pairing is gone. A query for "a review by ana with rating 2" has ana in the author list and 2 in the rating list, so the document matches, even though ana gave a 5.

JS The flattening problem

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The flattened check says true while the truth is false. Whenever the combination of fields inside one list item matters, object is the wrong type.

The nested type

Mapping the field as nested tells Elasticsearch to index each array element as its own hidden Lucene document, kept next to its parent. Fields inside one element stay together.

HTTP
PUT /products
{
  "mappings": {
    "properties": {
      "title": { "type": "text" },
      "reviews": {
        "type": "nested",
        "properties": {
          "author": { "type": "keyword" },
          "rating": { "type": "integer" }
        }
      }
    }
  }
}

You must then use a nested query, naming the path:

HTTP
GET /products/_search
{
  "query": {
    "nested": {
      "path": "reviews",
      "query": {
        "bool": {
          "must": [
            { "term":  { "reviews.author": "ana" } },
            { "range": { "reviews.rating": { "gte": 4 } } }
          ]
        }
      }
    }
  }
}

Inside the inner query, both conditions must hold for the same review. A plain match on reviews.author outside a nested query would find nothing, because the elements are hidden documents. Aggregations over nested data need a matching nested aggregation, and inner_hits returns which reviews matched.

JS Nested matching per element

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Each review is tested alone, so the Trail shoe no longer matches "ana with rating 2", and "bo with 5" finds nothing. This is how a real nested query treats each element.

The cost of nested

Nested is correct but not free:

  • Every element is a separate Lucene document, so a product with 100 reviews is 101 documents. There are safety limits: index.mapping.nested_fields.limit (50 nested fields per index) and index.mapping.nested_objects.limit (10,000 nested objects per document).
  • Updating one nested element reindexes the whole parent document and all its elements.
  • Nested queries and aggregations are slower than flat ones.

Use nested only when you must query combinations of fields inside list items and the lists are modest in size.

The join field: parent and child

A join field lets you index parents and children as separate documents in the same index, linked by a relation. Children can be updated without touching the parent.

HTTP
PUT /qa
{
  "mappings": {
    "properties": {
      "text": { "type": "text" },
      "relation": { "type": "join", "relations": { "question": "answer" } }
    }
  }
}

PUT /qa/_doc/q1
{ "text": "How do I rename an index?", "relation": "question" }

PUT /qa/_doc/a1?routing=q1
{ "text": "Reindex it.", "relation": { "name": "answer", "parent": "q1" } }

The trade-offs are significant. Parent and children must live on the same shard (so you pass routing), only one join relation per index is recommended, and has_child/has_parent queries are noticeably slower than nested queries. Use it only when children are numerous and change often while parents rarely do.

Often the best answer: denormalize

Both nested and join exist to preserve relationships. Frequently you do not need them. If users search for products and only need "average rating" or "number of reviews", store those numbers directly on the product and keep the reviews elsewhere. If you need one fixed combination, add a flat helper field (for example reviews.author_rating as "ana:5" in a keyword array). The next lesson, Data Modeling and Denormalization, builds on this.

Option Keeps pairing Update cost Query cost
object (default) No Low Low
nested Yes Reindexes parent Medium
join Yes Child only High
Denormalized fields Not needed Depends on duplication Low

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. A product has reviews [{author: ana, rating: 5}, {author: bo, rating: 2}] mapped as a plain object. Why does a query for author ana AND rating 2 match?

    Flattening gives reviews.author [ana, bo] and reviews.rating [5, 2]. The pairing is lost, so both conditions are satisfied by different elements.
  2. 2. How do you query a nested field?

    Nested elements are hidden documents, so a nested query with a path is required.
  3. 3. What happens when you update one review inside a nested array?

    Documents are immutable in segments, so the parent is rewritten along with every nested element.
  4. 4. When is a join field a reasonable choice?

    Join avoids reindexing parents, but it needs shared-shard routing and slower queries, so use it only for that kind of workload.

Summary

  • Objects are flattened to dotted field names; arrays of values match if any value matches.
  • In an array of objects, the link between fields of one element is lost, which causes false matches.
  • The nested type indexes each element as a hidden document and needs nested queries; it costs more to store, update and query.
  • A join field keeps parents and children as separate documents on the same shard, with slower queries.
  • Prefer denormalization when a flat copy of the needed data is enough.

Next: Data Modeling and Denormalization.