Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 39 ยท Caching and Performance

Batch Processing and Hibernate Statistics

Insert and update thousands of rows efficiently with JDBC batching, flush-and-clear loops and StatelessSession, and measure queries with Hibernate statistics.

Hibernate is happy with a handful of entities per transaction. Import 100,000 rows with a naive loop and you meet two problems: the persistence context grows until memory runs out, and every insert is a separate network round trip. This lesson shows the standard fixes (JDBC batching, periodic flush and clear, StatelessSession) and how to measure with Hibernate statistics so you tune what is really slow.

Hibernate is not available in the browser, so its configuration and session code is read-only. The runnable programs use H2 and plain JDBC to demonstrate batching and counting, which are the same ideas Hibernate relies on.

Why a naive loop is slow

Java
for (int i = 0; i < 100_000; i++) {
    session.persist(new Reading(i, value(i)));
}

Every persisted entity stays in the first-level cache until the transaction ends, and dirty checking at flush time compares all of them. With GenerationType.IDENTITY ids Hibernate must also insert each row immediately to learn its id, so JDBC batching is disabled. Choose SEQUENCE (with a pooled optimizer) for entities you insert in bulk.

Turn on JDBC batching

PROPERTIES
hibernate.jdbc.batch_size=50
hibernate.order_inserts=true
hibernate.order_updates=true
hibernate.jdbc.batch_versioned_data=true

With batch_size=50, Hibernate collects up to 50 identical statements and sends them to the driver in one executeBatch(). order_inserts and order_updates sort statements by entity type so they can be grouped; without them, alternating parent and child inserts break batches into tiny pieces. Some drivers need an extra flag to really batch (for example reWriteBatchedInserts=true on PostgreSQL, rewriteBatchedStatements=true on MySQL).

Flush and clear in slices

Java
Transaction tx = session.beginTransaction();
for (int i = 0; i < 100_000; i++) {
    session.persist(new Reading(i, value(i)));
    if (i % 50 == 49) {          // same number as batch_size
        session.flush();          // send the batch
        session.clear();          // forget the entities
    }
}
tx.commit();

flush writes the pending statements, clear empties the persistence context so memory stays flat. The slice size should match hibernate.jdbc.batch_size. For updates read in chunks, do not use plain list() on a huge result; use scroll() (or stream()) so rows arrive one at a time:

Java
try (ScrollableResults<Reading> rows = session.createQuery("from Reading r", Reading.class)
        .setFetchSize(100).scroll(ScrollMode.FORWARD_ONLY)) {
    int n = 0;
    while (rows.next()) {
        rows.get().recalculate();
        if (++n % 50 == 0) { session.flush(); session.clear(); }
    }
}

StatelessSession

A StatelessSession has no persistence context, no dirty checking, no lazy loading and no cache. You call insert, update, delete and every call is an immediate SQL statement.

Java
try (StatelessSession ss = sessionFactory.openStatelessSession()) {
    Transaction tx = ss.beginTransaction();
    for (Reading r : readings) { ss.insert(r); }
    tx.commit();
}

It is the leanest way to load data, but it gives up cascades, callbacks and the identity guarantee, so use it for imports and reports and not for domain logic.

Java One round trip per row versus a JDBC batch (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Hibernate statistics

Guessing about performance wastes time. Turn on statistics in a test or a staging run:

PROPERTIES
hibernate.generate_statistics=true
hibernate.session.events.log.LOG_QUERIES_SLOWER_THAN_MS=25
Java
Statistics stats = sessionFactory.getStatistics();
stats.clear();
runUseCase();
System.out.println("statements:  " + stats.getPrepareStatementCount());
System.out.println("queries:     " + stats.getQueryExecutionCount());
System.out.println("entity loads:" + stats.getEntityLoadCount());
System.out.println("L2 hits:     " + stats.getSecondLevelCacheHitCount());
System.out.println("slowest:     " + stats.getQueryExecutionMaxTimeQueryString());

Useful counters: statements prepared, entities loaded/inserted/updated, collection fetches, cache hits and misses, flush count and the slowest query. Compare numbers before and after a change instead of trusting intuition. In production, expose them through Micrometer or JMX, but note the small overhead.

Another trick is a "statement counter" in tests: assert that a use case issues at most N statements, which turns an N+1 regression into a failing test (see Testing Hibernate Code).

Java A tiny statistics counter around JDBC (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Why does GenerationType.IDENTITY prevent insert batching?

    With IDENTITY the id only exists after the INSERT ran, so Hibernate cannot postpone and group the statements. Sequences let it assign ids first.
  2. 2. What is the purpose of session.clear() inside a long import loop?

    Without clear, every persisted entity stays in the persistence context until the transaction ends.
  3. 3. Which statement about StatelessSession is true?

    StatelessSession trades features for speed: no first-level cache, no dirty checking, no cascades, no lazy loading.
  4. 4. What do hibernate.order_inserts and order_updates do?

    Ordering avoids alternating between tables, which would break a batch into many small ones.

Summary

  • A loop of persist calls is slow and memory hungry; enable hibernate.jdbc.batch_size and order inserts and updates.
  • Use sequence ids, flush and clear every batch, and scroll large result sets.
  • StatelessSession is the lean option for imports and reports.
  • Turn on Hibernate statistics (or a statement counter) and measure before tuning.

Next: The Second-Level Cache