Skip to content
Tutorials Center
My learning

Topics

Lesson 36 of 43 ยท Working from Code

Elasticsearch from Java and Spring Data Elasticsearch

Use Elasticsearch from Java: the official Java API client for index, get, search and delete, and Spring Data Elasticsearch repositories with mapping annotations and derived queries.

Java is one of the most common languages around Elasticsearch, and two libraries cover most needs. The Elasticsearch Java API client is the official, low-level-but-typed client: it mirrors the REST API with builders for every request. Spring Data Elasticsearch sits on top of it for Spring applications and gives you repositories, annotations and derived queries, much like Spring Data JPA does for relational databases.

This lesson shows both. The cluster cannot run on this site, so the Java and Spring code is read-only. Library and API details change between versions, so check the documentation for the version you use. The TryIt blocks are JavaScript stand-ins that copy the ideas, a request builder and a repository that derives queries from method names. They are not the real libraries.

Two ways to talk to Elasticsearch from Java

Java API client Typed builders for every REST call.
Spring Data Repositories and annotations.
HTTP and JSON Both end up sending plain REST.

Use the client for full control and Spring Data for simple CRUD and search.

1 / 2

The Java API client

Add the dependency (for example co.elastic.clients:elasticsearch-java, with a JSON mapper such as Jackson) and create a client from a low-level REST transport:

Java
RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200, "https"))
    .setDefaultHeaders(new Header[] {
        new BasicHeader("Authorization", "ApiKey " + System.getenv("ES_API_KEY"))
    })
    .build();

ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());
ElasticsearchClient client = new ElasticsearchClient(transport);

Use an API key with narrow rights, as described in security and operations. Create the client once and share it, because it holds a connection pool. Close the transport on shutdown.

Documents are ordinary Java classes. A simple record works with Jackson:

Java
public record Product(String sku, String name, double price, String category) {}

Index, get and delete

Java
Product shoe = new Product("sku-1", "Trail shoe", 89.0, "footwear");

IndexResponse indexed = client.index(i -> i
    .index("products")
    .id(shoe.sku())
    .document(shoe));
System.out.println(indexed.result());          // Created or Updated

GetResponse<Product> got = client.get(g -> g.index("products").id("sku-1"), Product.class);
if (got.found()) {
    System.out.println(got.source().name());
}

client.delete(d -> d.index("products").id("sku-1"));

The lambdas build request objects fluently (i -> i.index(...)), and the second argument of get tells the client which class to decode _source into.

Java
SearchResponse<Product> response = client.search(s -> s
    .index("products")
    .query(q -> q.bool(b -> b
        .must(m -> m.match(t -> t.field("name").query("trail shoe")))
        .filter(f -> f.range(r -> r.number(n -> n.field("price").lte(120.0))))))
    .size(10),
    Product.class);

for (Hit<Product> hit : response.hits().hits()) {
    System.out.println(hit.score() + " " + hit.source().name());
}

This is the same bool query you wrote in compound queries, expressed with builders. Aggregations work the same way (.aggregations("by_cat", a -> a.terms(t -> t.field("category")))) and the response has typed accessors for buckets.

Bulk and errors

Java
BulkRequest.Builder br = new BulkRequest.Builder();
for (Product p : products) {
    br.operations(op -> op.index(i -> i.index("products").id(p.sku()).document(p)));
}
BulkResponse result = client.bulk(br.build());

if (result.errors()) {
    result.items().stream()
        .filter(item -> item.error() != null)
        .forEach(item -> System.err.println(item.id() + ": " + item.error().reason()));
}

As with every client, a bulk call can succeed overall while single items fail, so always inspect errors() (clients and bulk). A failed call throws ElasticsearchException for errors returned by the server (such as version_conflict_engine_exception) and IOException for network problems.

A builder stand-in

JavaScript has no such fluent lambdas, so the stand-in below builds the same request body with small functions. It shows the idea: the builder produces plain JSON, and nothing magic happens on the wire.

JS Builders that produce a search body

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Spring Data Elasticsearch

Spring Data Elasticsearch (and the Spring Boot starter spring-boot-starter-data-elasticsearch) lets you describe your documents with annotations and get repositories for free. Connection settings go into application.properties, for example spring.elasticsearch.uris=https://localhost:9200 plus credentials.

Mapping with annotations

Java
@Document(indexName = "products")
public class Product {

    @Id
    private String sku;

    @Field(type = FieldType.Text, analyzer = "english")
    private String name;

    @Field(type = FieldType.Keyword)
    private String category;

    @Field(type = FieldType.Double)
    private double price;

    @Field(type = FieldType.Date, format = DateFormat.date_time)
    private Instant createdAt;

    // constructor, getters, setters
}

@Document names the index, @Id marks the document id, and @Field sets the field type and analyzer. This is where you decide text versus keyword, the choice that matters most for searching. Spring Data can create the index and its mapping from these annotations at startup (createIndex = true, the default), which is handy in development. In production many teams create indexes through templates or migrations instead and keep the annotations in sync.

Repositories and derived queries

Java
public interface ProductRepository extends ElasticsearchRepository<Product, String> {

    List<Product> findByName(String name);

    List<Product> findByCategoryAndPriceLessThan(String category, double maxPrice);

    Page<Product> findByCategory(String category, Pageable pageable);

    @Query("{\"match\": {\"name\": {\"query\": \"?0\"}}}")
    List<Product> searchByText(String text);
}

With no implementation written, you already get save, saveAll, findById, deleteById, count and findAll. Method names such as findByCategoryAndPriceLessThan are parsed into queries: a term-style condition on category, a range on price, joined with and. For a text field, findByName becomes a match. When names are not enough, @Query takes a JSON query body with ?0 placeholders for arguments.

Java
@Service
public class CatalogService {
    private final ProductRepository repository;

    public CatalogService(ProductRepository repository) {
        this.repository = repository;
    }

    public Page<Product> shoesUnder(double max, int page) {
        return repository.findByCategory("footwear", PageRequest.of(page, 20, Sort.by("price")));
    }
}

For anything beyond that, such as aggregations or complex scoring, inject ElasticsearchOperations and build a native query with the same builders as the Java client.

A repository stand-in

The stand-in keeps documents in memory and turns method names such as findByCategoryAndPriceLessThan into filters. It is a tiny imitation of the name parser.

JS Derived query methods

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The real parser supports many more keywords (Between, In, Containing, OrderBy, Not), and it generates Elasticsearch queries instead of filtering a list.

Practical tips

  • Pick one: the client for control over queries and aggregations, Spring Data for CRUD-like use. They combine well, because Spring Data's ElasticsearchOperations can run client-style queries.
  • Match versions: the client and Spring Data release should match your cluster's major version.
  • Do not model Elasticsearch as your system of record. Keep the source of truth in a database and index copies, as discussed in modeling data.
  • Reindex on mapping changes. Annotations cannot change the mapping of an existing field. Create a new index and switch an alias, as shown in clients and bulk.
  • Test against a real node. Mocks cannot show analyzer behaviour. Tests with a throwaway container are more trustworthy.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does the second argument of client.get(..., Product.class) tell the Java client?

    The client deserializes the document source into the class you pass.
  2. 2. In Spring Data Elasticsearch, which annotation sets a field's type and analyzer?

    @Field carries the field type, analyzer and format. @Document names the index.
  3. 3. How is findByCategoryAndPriceLessThan turned into a query?

    The method name is parsed into a term-style condition and a range, combined with AND.
  4. 4. A bulk call returns without throwing. What should you still check?

    Individual operations in a bulk request can fail while the HTTP call succeeds.

Summary

  • The Elasticsearch Java API client mirrors the REST API with typed builders; create one shared client with an API key and close it on shutdown.
  • index, get, search, delete and bulk take lambdas that build the same JSON you send from Dev Tools, and bulk results must be checked item by item.
  • Spring Data Elasticsearch adds @Document, @Id and @Field mapping annotations plus repositories with derived and @Query methods.
  • Use ElasticsearchOperations for aggregations and complex queries, keep client and cluster versions aligned, and change mappings through a new index and an alias.

Next: protect the data your application reads and writes in security, backups and operations.