This is a quick tutorial on stating the difference between Executors.newCachedThreadPool() and Executors.newFixedThreadPool() in Java MultiThreading.

The ThreadPoolExecutor class is the base implementation for the executors that are returned from many of the Executors factory methods. So let’s approach Fixed and Cached thread pools from ThreadPoolExecutor’s perspective.

ThreadPoolExecutor

The main constructor of this class looks like this:

public ThreadPoolExecutor(
                  int corePoolSize,
                  int maximumPoolSize,
                  long keepAliveTime,
                  TimeUnit unit,
                  BlockingQueue<Runnable> workQueue,
                  ThreadFactory threadFactory,
                  RejectedExecutionHandler handler
)

Core Pool Size

The corePoolSize determines the minimum size of the target thread pool. The implementation would maintain a pool of that size even if there are no tasks to execute.

Maximum Pool Size

The maximumPoolSize is the maximum number of threads that can be active at once.

After the thread pool grows and becomes bigger than the corePoolSize threshold, the executor can terminate idle threads and reach to the corePoolSize again. If allowCoreThreadTimeOut is true, then the executor can even terminate core pool threads if they were idle more than keepAliveTime threshold.

So the bottom line is if threads remain idle more than keepAliveTime threshold, they may get terminated since there is no demand for them.

Queuing

What happens when a new task comes in and all core threads are occupied?

The new tasks will be queued inside that BlockingQueue instance. When a thread becomes free, one of those queued tasks can be processed.

There are different implementations of the BlockingQueue interface in Java, so we can implement different queuing approaches like:

Bounded Queue: New tasks would be queued inside a bounded task queue.

Unbounded Queue: New tasks would be queued inside an unbounded task queue. So this queue can grow as much as the heap size allows.

Synchronous Handoff: We can also use the SynchronousQueue to queue the new tasks. In that case, when queuing a new task, another thread must already be waiting for that task.

Work Submission

Here’s how the ThreadPoolExecutor executes a new task:

1. If fewer than corePoolSize threads are running, tries to start a new thread with the given task as its first job.
2. Otherwise, it tries to enqueue the new task using the BlockingQueue#offer method. The offer method won’t block if the queue is full and immediately returns false.
3. If it fails to queue the new task (i.e. offer returns false), then it tries to add a new thread to the thread pool with this task as its first job.
4. If it fails to add the new thread, then the executor is either shut down or saturated. Either way, the new task would be rejected using the provided RejectedExecutionHandler.

The main difference between the fixed and cached thread pools boils down to these three factors:

1. Core Pool Size
2. Maximum Pool Size

3. Queuing

   +-----------+-----------+-------------------+---------------------------------+
   | Pool Type | Core Size |    Maximum Size   |         Queuing Strategy        |
   +-----------+-----------+-------------------+---------------------------------+
   |   Fixed   | n (fixed) |     n (fixed)     | Unbounded `LinkedBlockingQueue` |
   +-----------+-----------+-------------------+---------------------------------+
   |   Cached  |     0     | Integer.MAX_VALUE |        `SynchronousQueue`       |
   +-----------+-----------+-------------------+---------------------------------+

   Fixed Thread Pool

   Here’s how the Excutors.newFixedThreadPool(n) works:

   public static ExecutorService newFixedThreadPool(int nThreads) {
   return new ThreadPoolExecutor(nThreads, nThreads,
                                 0L, TimeUnit.MILLISECONDS,
                                 new LinkedBlockingQueue<Runnable>());
   }

   As you can see:

4. The thread pool size is fixed.
5. If there is high demand, it won’t grow.
6. If threads are idle for quite some time, it won’t shrink.
7. Suppose all those threads are occupied with some long-running tasks and the arrival rate is still pretty high. Since the executor is using an unbounded queue, it may consume a huge part of the heap. Being unfortunate enough, we may experience an OutOfMemoryError.

When should I use one or the other? Which strategy is better in terms of resource utilization?

A fixed-size thread pool seems to be a good candidate when we’re going to limit the number of concurrent tasks for resource management purposes.

For example, if we’re going to use an executor to handle web server requests, a fixed executor can handle the request bursts more reasonably.

For even better resource management, it’s highly recommended to create a custom ThreadPoolExecutor with a bounded BlockingQueue implementation coupled with reasonable RejectedExecutionHandler.

Cached Thread Pool

Here’s how the Executors.newCachedThreadPool() works:

public static ExecutorService newCachedThreadPool() {
    return new ThreadPoolExecutor(0, Integer.MAX_VALUE,
                                  60L, TimeUnit.SECONDS,
                                  new SynchronousQueue<Runnable>());
}

As you can see:

• The thread pool can grow from zero threads to Integer.MAX_VALUE. Practically, the thread pool is unbounded.
• If any thread is idle for more than 1 minute, it may get terminated. So the pool can shrink if threads remain too much idle.
• If all allocated threads are occupied while a new task comes in, then it creates a new thread, as offering a new task to a SynchronousQueue always fails when there is no one on the other end to accept it!

When should I use one or the other? Which strategy is better in terms of resource utilization?

Use it when you have a lot of predictable short-running tasks.