Skip to content
Tutorials Center
My learning

Topics

Lesson 132 of 201 ยท Concurrency

Executors and Thread Pools

Learn Java executors - ExecutorService, fixed, cached and scheduled thread pools, Callable and Future, invokeAll and invokeAny, and how to shut a pool down properly.

Creating a new Thread for every task works in a demo, but it doesn't scale. Each platform thread costs memory for its stack and time to start, and nothing stops a busy server from creating ten thousand of them and falling over. Professional Java code almost never manages threads by hand. Instead it hands tasks to an executor, which owns a small, reusable set of threads called a thread pool.

This separates what to run (a Runnable or Callable) from how it runs (how many threads, which queue, what happens under overload). In this lesson you'll create the standard pools, get results back with Callable and Future, run batches with invokeAll and invokeAny, schedule repeating work, configure a ThreadPoolExecutor directly, and shut everything down the right way. It builds on Threads and Atomic Variables.

Your first ExecutorService

An everyday picture. Think of a restaurant with a fixed team of waiters. Customers (tasks) arrive in any number; there is no point hiring and firing a waiter for each customer, because that takes longer than serving the meal. Instead the same waiters (threads) are reused for customer after customer, and when all of them are busy the new customers wait in a queue. That is a thread pool: it saves the cost of creating threads, reuses them, limits how many run at once (so a burst of requests can't use all memory), and gives you a queue and a single place to shut everything down.

Executors is a factory class. newFixedThreadPool(n) creates a pool with exactly n threads and a queue for waiting tasks. You submit tasks; idle threads take them from the queue.

Java Five tasks, two threads

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Only two thread names appear (pool-1-thread-1 and pool-1-thread-2), reused for all five tasks. The exact pairing of tasks to threads, and the order of the lines, can change between runs. "all tasks submitted" usually prints early because submit doesn't wait; it only queues the task.

Since Java 19, ExecutorService implements AutoCloseable, so try-with-resources shuts the pool down and waits for running and queued tasks to finish. That's the cleanest pattern for a pool with a clear lifetime.

The standard pools

Factory method Threads Best for
newFixedThreadPool(n) exactly n, kept alive CPU-bound work; a predictable limit
newCachedThreadPool() grows without limit, idle threads die after 60 s many short tasks with low, bursty load
newSingleThreadExecutor() one running tasks strictly one at a time, in order
newScheduledThreadPool(n) n delayed and repeating tasks
newWorkStealingPool() a ForkJoinPool sized to the CPU count many small independent tasks (Fork/Join)
newVirtualThreadPerTaskExecutor() a new virtual thread per task (Java 21) huge numbers of blocking I/O tasks (Virtual Threads)

Two traps hide in the convenience methods. newFixedThreadPool uses an unbounded queue, so if tasks arrive faster than they finish, the queue grows until memory runs out. newCachedThreadPool has an unbounded number of threads, so a flood of slow tasks creates a flood of threads. For servers, you'll often configure a ThreadPoolExecutor yourself, as shown later.

How many threads? For CPU-bound work, about one per core (Runtime.getRuntime().availableProcessors()): more threads just fight over the same cores. For blocking I/O work, threads spend most of their time waiting, so you need more, which is exactly the case virtual threads solve better.

Callable and Future: getting results back

A Runnable returns nothing and can't throw checked exceptions. A Callable<V> returns a V and may throw. submit(callable) returns a Future<V>, a handle to a result that may not exist yet.

Java Callable, Future and get

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

The Future methods you'll use:

  • get() waits for the result; get(timeout, unit) waits at most that long and throws TimeoutException.
  • An exception thrown by the task is wrapped in an ExecutionException; getCause() gives you the original.
  • isDone() is true once the task finished in any way (success, failure or cancellation).
  • cancel(true) tries to stop the task by interrupting its thread; cancel(false) only prevents it from starting.
  • Java 19 added state() (RUNNING, SUCCESS, FAILED, CANCELLED), resultNow() and exceptionNow() for tasks that are known to be finished.

Notice the timed-out task in the example: the wait ended, but the task kept running in the pool. Closing the pool at the end waits for it. If you no longer need it, call slow.cancel(true).

execute(runnable) and submit(runnable) handle exceptions differently. With execute, an exception kills the worker thread and is printed by the default handler. With submit, the exception is stored inside the Future; if nobody calls get(), it disappears without a trace. Always check the futures you get back, or catch and log inside the task.

invokeAll and invokeAny

For a batch of Callables, invokeAll submits them all and waits until every one is done, returning the futures in the same order as the tasks. invokeAny returns the result of the first one to succeed and cancels the rest, which is handy for asking several replicas and taking the fastest answer.

Java invokeAll and invokeAny

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

invokeAll printed results in task order (paris, tokyo, broken, lima), not in finishing order, and took as long as the slowest task, about 300 ms. invokeAny skipped the failing server, which finished first, and returned Tokyo's answer, the fastest successful one. Both methods also have overloads with a timeout.

Shutting down properly

A pool's threads are non-daemon, so a program whose pool is never shut down never exits. When you can't use try-with-resources (a pool that lives as long as your application, for example), use the classic sequence:

Java
pool.shutdown();                                   // stop accepting new tasks; queued ones still run
try {
    if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
        pool.shutdownNow();                        // interrupt running tasks, drop the queue
        if (!pool.awaitTermination(5, TimeUnit.SECONDS)) {
            System.err.println("pool did not terminate");
        }
    }
} catch (InterruptedException e) {
    pool.shutdownNow();
    Thread.currentThread().interrupt();
}
  • shutdown() is gentle: no new tasks (submitting one throws RejectedExecutionException), but everything already queued finishes.
  • shutdownNow() interrupts running tasks and returns the list of tasks that never started. Interruption is only a request: a task stops only if it checks Thread.interrupted() or calls a blocking method that throws InterruptedException (Thread Lifecycle and Priorities).
  • awaitTermination blocks until the pool is empty or the timeout passes.
  • close() (Java 19+) is shutdown() followed by waiting until termination.

Scheduled executors

ScheduledExecutorService runs tasks after a delay or repeatedly. There are two ways to repeat:

  • scheduleAtFixedRate(task, initialDelay, period, unit) aims for a fixed start-to-start time, like a clock tick.
  • scheduleWithFixedDelay(task, initialDelay, delay, unit) waits a fixed time after each run ends, so slow runs push the next one back.
Java Delayed and repeating tasks

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

You'll see about five ticks, roughly 100 ms apart, with the one-shot task slotted in between; the exact count and timings wobble a little. Two rules matter in real code: a repeating task runs until you cancel it or shut down the scheduler, and if one run throws an exception, all later runs are silently cancelled. Wrap the body of periodic tasks in try/catch and log failures.

ThreadPoolExecutor: full control

All the fixed and cached pools are instances of ThreadPoolExecutor with particular settings. Creating one yourself lets you bound everything:

Java
new ThreadPoolExecutor(
    corePoolSize,        // threads kept alive even when idle
    maximumPoolSize,     // upper limit, used only when the queue is full
    keepAliveTime, unit, // how long extra threads stay idle before dying
    workQueue,           // where waiting tasks go (bounded = backpressure)
    threadFactory,       // names and creates threads
    rejectionHandler);   // what to do when both threads and queue are full

A new task goes to an idle core thread if there is one, otherwise into the queue, and only when the queue is full does the pool start extra threads up to the maximum. When even that is full, the rejection policy decides: AbortPolicy (the default) throws RejectedExecutionException, CallerRunsPolicy runs the task on the submitting thread (which naturally slows the producer down), DiscardPolicy drops it silently, and DiscardOldestPolicy drops the oldest queued task.

Java A bounded pool that rejects work

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with IO.readln() or a Scanner), it gets the next line from this box.

Trace it: task 1 starts the core thread, tasks 2 and 3 fill the queue, task 4 finds the queue full and gets a second thread, and tasks 5 and 6 are rejected because both the threads and the queue are at their limits. Four tasks complete. A custom ThreadFactory built with Thread.ofPlatform().name("worker-", 1) gives threads meaningful names (worker-1, worker-2), which is a gift when you read a thread dump at 3 a.m.

Good habits with executors

  • Bound your queues in servers, and decide consciously what happens under overload.
  • Never block a pool on its own tasks. If a task in a fixed pool of 2 submits another task to the same pool and waits for it, two such tasks can wait forever for a thread that will never be free (thread-starvation deadlock).
  • Pass the interrupt on. When you catch InterruptedException in a task and can't rethrow it, call Thread.currentThread().interrupt() so shutdownNow still works.
  • Don't leak ThreadLocal state. Pool threads are reused, so a ThreadLocal set by one task is still there for the next one unless you remove() it.
  • Keep results flowing. Blocking on get() right after submit() gives you a thread pool with none of the concurrency. For chains of async steps, use CompletableFuture.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A task submitted with submit() throws a RuntimeException, and the program never calls get() on its Future. What happens?

    submit captures exceptions in the Future. Only get() (wrapped in ExecutionException) or exceptionNow() reveals them, so unchecked futures hide failures.
  2. 2. What does future.get() throw when the task itself threw IllegalStateException?

    Future.get wraps task failures in ExecutionException. Call getCause() to reach the original exception.
  3. 3. A ThreadPoolExecutor has core size 1, max size 3 and an ArrayBlockingQueue of capacity 10. Ten slow tasks are submitted at once. How many threads does it run?

    The pool adds threads beyond the core size only when the queue is full. Task 1 takes the core thread and tasks 2 to 10 fit in the queue, so it stays at one thread.
  4. 4. Which statement about shutdown() is true?

    shutdown is the gentle stop and returns immediately. shutdownNow interrupts and returns unstarted tasks; awaitTermination or close() does the waiting.
  5. 5. A scheduleAtFixedRate task throws an exception on its third run. What happens next?

    A periodic task that throws is cancelled; the exception is only visible through its ScheduledFuture. Catch and log exceptions inside periodic tasks.

Summary

  • Executors separate tasks from threads; a thread pool reuses a few threads for many tasks.
  • Executors.newFixedThreadPool, newCachedThreadPool, newSingleThreadExecutor, newScheduledThreadPool and newVirtualThreadPerTaskExecutor cover most needs, but watch for unbounded queues and unbounded thread counts.
  • submit(Callable) returns a Future: get (with a timeout) waits, task exceptions arrive wrapped in ExecutionException, and state/resultNow (Java 19) inspect finished tasks.
  • invokeAll waits for a whole batch in order; invokeAny returns the first successful result.
  • Shut pools down: try-with-resources (close(), Java 19+), or shutdown + awaitTermination + shutdownNow.
  • Scheduled tasks repeat until cancelled; an exception stops them silently.
  • ThreadPoolExecutor gives full control over core/max threads, queue bounds, thread names and rejection policies.

Next: CompletableFuture, where you'll chain and combine asynchronous steps without blocking.