Skip to main content

6.4 Executors & Thread Pools: Architecture, Sizing & Shutdown

← Back to Chapter 6: Multithreading & Concurrency

Manually allocating new Thread() in server applications wastes 1 MB of stack memory per thread and risks OutOfMemoryError: unable to create native thread. The Executor Framework manages reusable worker thread pools.

What is Thread Pool?​

A Thread Pool is simply a collection of already-created threads that can execute tasks.

Suppose you have 4 threads but 10 tasks.

// first 4 tasks will get executed by 4 threads
Thread 1 → Task A
Thread 2 → Task B
Thread 3 → Task C
Thread 4 → Task D
// rest of the task will wait in the queue
Task E
Task F
Task G

When Thread 1 finishes Task A:

Thread 1 → Task E

So threads are reused instead of continuously creating new ones.

The remaining tasks wait in a queue.

Create thread → execute task → destroy thread ❌
Create threads once
       ↓
Execute task
       ↓
Reuse thread
       ↓
Execute another task
       ↓
Reuse again ✅

What is ExecutorService?​

ExecutorService is a Java interface that provides a way to create and manage thread pools and submit tasks to them.

Thread Pool = Workers

ExecutorService = Manager

Why we need Thread Pool? Why can't we simply do new Thread()?​

Imagine your application receives 1000 tasks. You could end up creating:

1000 tasks
   ↓
1000 threads

That's inefficient because threads consume memory and CPU resources. Therefore :

1000 tasks
   ↓
10 worker threads
   ↓
Tasks wait in queue
   ↓
Threads continuously process them

This is much better way.

Different common Thread Pools :​

1. newFixedThreadPool()​

Creates a fixed number of threads. Useful when you want controlled concurrency.

ExecutorService executor = Executors.newFixedThreadPool(5);

2. newCachedThreadPool()​

Creates threads as needed and can reuse idle threads.Good for many short-lived asynchronous tasks, but you need to be careful because it can create many threads under heavy load.

ExecutorService executor = Executors.newCachedThreadPool();

3. newSingleThreadExecutor()​

Only one worker thread. Tasks execute one after another.

ExecutorService executor = Executors.newSingleThreadExecutor();
100 requests
     ↓
Controller
     ↓
Service
     ↓
Some slow operation

Instead of creating a new thread for every operation, you can configure a thread pool and let Spring manage it.

For example, with @Async:

@Async
public void sendEmail() {
    // This can execute in a separate thread
}

Spring can use a configured executor/thread pool to execute these asynchronous tasks.

HTTP Request
     ↓
Spring Boot
     ↓
@Async method
     ↓
Executor
     ↓
Thread Pool
 ┌────┬────┬────┐
 T1   T2   T3   T4

Thread Pool Sizing Formulas​

  • CPU-Intensive Tasks (Encryption, Video compression, Complex JSON parsing): 
    Pool Size = CPU Cores + 1
  • I/O-Intensive Tasks (Database queries, REST API downstream calls, disk I/O): 
    Pool Size = CPU Cores * (1 + Wait Time / Compute Time)

4. Graceful Shutdown Protocol​

executor.shutdown(); // Disable new task admissions
try {
    if (!executor.awaitTermination(30, TimeUnit.SECONDS)) {
        executor.shutdownNow(); // Cancel executing tasks via interrupt
        if (!executor.awaitTermination(10, TimeUnit.SECONDS)) {
            System.err.println("Thread pool failed to terminate");
        }
    }
} catch (InterruptedException ie) {
    executor.shutdownNow();
    Thread.currentThread().interrupt();
}