JAVA / THREADS AND CONCURRENCY
ExecutorService and separating tasks from threads
Run many tasks on a small pool of reusable threads with ExecutorService, and shut that pool down cleanly so the JVM can exit.
What you will learn
- Submit work as Runnables to one ExecutorService instead of one Thread per task
- Predict how many threads a fixed pool creates and where extra tasks wait
- Close a pool with shutdown() plus awaitTermination() so the JVM can exit
- Choose shutdownNow() when queued work must be abandoned, not run
Understanding ExecutorService and separating tasks from threads
A Runnable is a description of work; a Thread is a machine that runs work. Writing new Thread(task).start() welds the two together: the worker exists to run exactly one task and dies with it, and every call site that wants concurrency also has to decide how many operating-system threads to create. An ExecutorService breaks that weld. You hand it tasks, it owns the threads, and the number of workers becomes a property of the pool instead of a side effect of how many times your code happened to call start().
A pool is less magic than it sounds: each worker thread sits in a loop taking the next task off a blocking queue and running it on its own call stack. Executors.newFixedThreadPool(4) means four such loops plus an unbounded LinkedBlockingQueue, which is why submitting 200 tasks does not create 200 threads, it just makes 196 of them stand in line. Queueing instead of spawning is what keeps memory and context switching flat under load, and it is equally the reason one slow task delays everything behind it rather than getting a thread of its own.
A pool is a resource with a lifecycle you are responsible for closing. Its threads are non-daemon by default, so a live pool keeps the JVM alive long after main returns. shutdown() closes the intake, meaning already-queued tasks still run while new submissions are rejected with RejectedExecutionException; shutdownNow() additionally drops the queue and interrupts running tasks; awaitTermination() is the only way to learn that the workers have actually finished, and its boolean return tells you whether you instead hit the timeout.
import java.util.Set;
import java.util.concurrent.ConcurrentSkipListSet;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
public class PoolBasics {
public static void main(String[] args) throws InterruptedException {
ExecutorService pool = Executors.newFixedThreadPool(2);
Set<String> workers = new ConcurrentSkipListSet<>();
AtomicInteger completed = new AtomicInteger();
for (int i = 0; i < 6; i++) {
pool.execute(() -> {
workers.add(Thread.currentThread().getName());
completed.incrementAndGet();
});
}
pool.shutdown();
boolean finished = pool.awaitTermination(2, TimeUnit.SECONDS);
System.out.println("tasks completed: " + completed.get());
System.out.println("distinct worker threads: " + workers.size());
System.out.println("worker names: " + workers);
System.out.println("terminated in time: " + finished);
}
}A task is what to do and a thread is who does it, and ExecutorService separates the two so threads become a bounded, reusable resource you configure once.
Worked examples
Six tasks, two threads, counted
Shows that a fixed pool answers 200 submissions with a fixed number of worker threads, while thread-per-task creates one thread per unit of work.
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class ReuseCount {
public static void main(String[] args) throws InterruptedException {
int tasks = 200;
Set<String> perTask = ConcurrentHashMap.newKeySet();
Set<String> pooled = ConcurrentHashMap.newKeySet();
Thread[] raw = new Thread[tasks];
for (int i = 0; i < tasks; i++) {
raw[i] = new Thread(() -> perTask.add(Thread.currentThread().getName()));
raw[i].start();
}
for (Thread t : raw) {
t.join();
}
ExecutorService pool = Executors.newFixedThreadPool(4);
for (int i = 0; i < tasks; i++) {
pool.execute(() -> pooled.add(Thread.currentThread().getName()));
}
pool.shutdown();
pool.awaitTermination(5, TimeUnit.SECONDS);
System.out.println("threads used by thread-per-task: " + perTask.size());
System.out.println("threads used by pool of 4: " + pooled.size());
}
}Example explained
Line 1Each new Thread gets a fresh default name (Thread-0, Thread-1, ...), so the set size equals the number of threads the JVM had to create and later tear down.
Line 2The pool grows only until it holds its core size of four workers; every submission after that lands in the pool's queue, so only four names ever appear.
Line 3The task bodies are identical in both halves, which is the point: the work did not change, only who runs it and how many workers exist.
Line 4awaitTermination is needed before reading pooled.size(), otherwise the main thread could print while workers are still draining the queue.
What shutdown() does and does not do
Demonstrates that a task submitted before shutdown() still runs, while a submission after it is rejected outright.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.RejectedExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;
public class ShutdownRules {
public static void main(String[] args) throws InterruptedException {
ExecutorService pool = Executors.newSingleThreadExecutor();
AtomicBoolean beforeRan = new AtomicBoolean();
AtomicBoolean afterRan = new AtomicBoolean();
pool.execute(() -> beforeRan.set(true));
pool.shutdown();
String verdict;
try {
pool.execute(() -> afterRan.set(true));
verdict = "accepted";
} catch (RejectedExecutionException e) {
verdict = "rejected";
}
boolean terminated = pool.awaitTermination(1, TimeUnit.SECONDS);
System.out.println("submit after shutdown: " + verdict);
System.out.println("task submitted before shutdown ran: " + beforeRan.get());
System.out.println("task submitted after shutdown ran: " + afterRan.get());
System.out.println("terminated: " + terminated);
}
}Example explained
Line 1shutdown() returns immediately and does not wait, yet the already-accepted task still executes, which is what orderly shutdown means.
Line 2The second execute() throws RejectedExecutionException because the default rejection policy is abort, and the pool is no longer in the running state.
Line 3awaitTermination returning true is the signal that the worker thread has exited; isShutdown() would already be true here even while work was still in flight.
Line 4The prints happen after termination so the output cannot interleave with the worker, which is why the ordering is stable.
A single-thread executor as a serialization point
Uses a one-thread pool so that an unsynchronized counter is still correct, because only one thread ever touches it.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class SerialWorker {
private static int counter = 0; // no lock, no volatile
public static void main(String[] args) throws InterruptedException {
ExecutorService worker = Executors.newSingleThreadExecutor();
for (int i = 0; i < 10_000; i++) {
worker.execute(() -> counter++);
}
worker.shutdown();
boolean terminated = worker.awaitTermination(5, TimeUnit.SECONDS);
System.out.println("terminated: " + terminated);
System.out.println("counter: " + counter);
}
}Example explained
Line 1newSingleThreadExecutor has exactly one worker and a FIFO queue, so the 10,000 tasks run one after another with no overlap.
Line 2counter++ needs no lock here only because every increment happens on that one worker thread; the field is confined to it, not protected by it.
Line 3The main thread's read of counter is safe because awaitTermination returned after the worker finished, which orders that read behind all the writes.
Line 4Hand the same field to a second executor and the race returns; the safety comes from the single thread, not from using an executor.
Sizing a pool for blocking work
Shows why a pool sized by CPU count starves when tasks spend their time waiting rather than computing.
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
public class Sizing {
static void fakeIo() {
try {
Thread.sleep(100); // stands in for a network call
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
static long runWith(int poolSize, int tasks) throws InterruptedException {
ExecutorService pool = Executors.newFixedThreadPool(poolSize);
long start = System.nanoTime();
for (int i = 0; i < tasks; i++) {
pool.execute(Sizing::fakeIo);
}
pool.shutdown();
pool.awaitTermination(1, TimeUnit.MINUTES);
return (System.nanoTime() - start) / 1_000_000;
}
public static void main(String[] args) throws InterruptedException {
long slow = runWith(2, 8);
long fast = runWith(8, 8);
System.out.println("8 blocking tasks on 2 threads: about " + (slow / 100) + " rounds of 100ms");
System.out.println("8 blocking tasks on 8 threads: about " + (fast / 100) + " rounds of 100ms");
}
}Example explained
Line 1With two workers the eight tasks are processed in four sequential batches, because a blocked worker still occupies its slot in the pool.
Line 2With eight workers all tasks wait at the same time, so total time collapses to roughly one task's duration.
Line 3Dividing elapsed milliseconds by 100 turns a timing measurement into a stable count of batches instead of a machine-dependent number.
Line 4This is why blocking work wants more threads than cores while CPU-bound work does not: extra threads only help when workers are idle rather than computing.
Important notes
execute() takes a Runnable and returns nothing, so a task that throws kills its worker thread and the pool quietly replaces it; submit() instead stores the failure in the returned Future, which is the subject of the next lesson.
Executors.newCachedThreadPool is unbounded in threads, not queue: a burst of slow tasks will create a thread per task, so prefer a fixed pool when you need an upper limit on concurrency.
Common mistakes
Creating a new ExecutorService inside a loop or per request: that is thread-per-task with extra bookkeeping, and because each pool is never shut down its non-daemon threads accumulate until the JVM throws OutOfMemoryError: unable to create native thread.
Never calling shutdown(): all output appears, main returns, and the program still hangs because the idle pool threads are non-daemon and keep the JVM alive.
Treating shutdown() as a stop button or skipping awaitTermination: shutdown() returns instantly while queued tasks keep running, so code after it reads half-finished results, and even shutdownNow() cannot stop a task that ignores interrupts.
Try it yourself
Change, predict, then run
In a browser editor, create Executors.newFixedThreadPool(3) and submit 12 tasks that each add Thread.currentThread().getName() to a ConcurrentHashMap.newKeySet(), then shutdown, awaitTermination and print the set. Change the pool size to 12, re-run, and confirm the number of names follows the pool size rather than the task count.
Open the Java workspaceCheck your understanding
You create Executors.newFixedThreadPool(3) and immediately execute 30 short Runnables. What happens?
- The pool creates 30 threads, one per task, and discards 27 of them when they finish
- The first 3 tasks run and the remaining 27 are rejected with RejectedExecutionException
- Three worker threads are created and the other 27 tasks wait in the pool's queue until a worker is free
- The pool temporarily grows beyond 3 threads because its queue is unbounded and it prefers new threads to waiting
Show answer
A fixed pool has the same core and maximum size, so it stops at three workers and parks everything else in an unbounded LinkedBlockingQueue, which those three threads drain in order. Rejection sounds plausible but only happens after shutdown or when a bounded queue is full and the pool is already at maximum size; an unbounded queue can never overflow into extra threads, which also rules out the last option.