CPU‑Bound and I/O‑Bound Tasks in Thread Pools | Concurrency Interview Question
- Posted
- 19 Feb 2026
Summary
This post details a common concurrency interview question regarding thread pool design for CPU-bound and I/O-bound tasks. It covers the problems arising from mixing them and proposes a strategy for optimal resource utilization with separate pools.
How it went
You are designing a backend service that needs to handle two types of workloads:
- CPU‑bound tasks (e.g., image transformations, number crunching).
- I/O‑bound tasks (e.g., database queries, network calls).
Initially, both types of tasks were submitted to the same thread pool. Over time, the system started showing:
- Unpredictable latency.
- Idle CPUs while threads waited on slow DB calls.
- SLAs slipping due to starvation of compute-heavy jobs.
Problems with a single pool:
- I/O tasks block threads, starving CPU tasks.
- CPUs remain underutilized while threads wait.
- Latency becomes unpredictable and tuning is difficult.
Solution:
- Use separate thread pools for CPU‑bound and I/O‑bound workloads.
- CPU pool: Size ≈ number of cores (or cores + 1). Keeps CPUs busy without excessive context switching.
- I/O pool: Larger, bounded pool. Prevents overwhelming downstream services while allowing concurrency for blocking calls.
ExecutorService cpuPool = Executors.newFixedThreadPool( Runtime.getRuntime().availableProcessors());ExecutorService ioPool = Executors.newFixedThreadPool(DEFAULT_IO_POOL_SIZE);
// CPU-bound task cpuPool.submit(() -> { // heavy computation });
// I/O-bound task ioPool.submit(() -> { // DB or network call });
Follow‑Up
- How would you monitor and adjust pool sizes in production?
- What risks exist if the I/O pool is unbounded?
👉 I break down such concurrecy concept step‑by‑step (with code) in my YT videos + LinkedIn posts. Check it out here: LINKED IN: linkedin.com/in/code-granular-43288739a/ YT : CodeGranular -> Breaking Down Java Concurrency Playlist
Whether you’re aiming for FAANG interviews or building enterprise-grade systems, understanding concurrency is not optional. It’s the difference between writing code that just runs and writing code that runs correctly, reliably, and at scale.
Questions asked (2)
- 01
Thread Pool Strategy for CPU-bound and I/O-bound Tasks
System Design · Medium
You are designing a backend service that needs to handle two types of workloads:
- CPU‑bound tasks (e.g., image transformations, number crunching).
- I/O‑bound tasks (e.g., database queries, network calls).
Initially, both types of tasks were submitted to the same thread pool. Over time, the system started showing:
- Unpredictable latency.
- Idle CPUs while threads waited on slow DB calls.
- SLAs slipping due to starvation of compute-heavy jobs.
What problems arise when mixing CPU‑bound and I/O‑bound tasks in the same thread pool?
How would you design the thread pool strategy to avoid these issues?
What sizing rules would you apply for each pool?
- 02
Monitoring and Adjusting Thread Pool Sizes
System Design · Hard
How would you monitor and adjust pool sizes in production? What risks exist if the I/O pool is unbounded?