It’s usually easier imo to separate them into different processes (…)
I don’t think your comment applies to the discussion. One of the thread pools mentioned is for IO-bound applications, which means things like sending HTTP requests.
Even if somehow you think it’s a good idea to move this class of tasks to a separate process, you will still have a very specific thread pool that can easily overcommit because most tasks end up idling while waiting for data to arrive.
The main take is that there are at least two classes of background tasks that have very distinct requirements and usage patterns. It’s important to handle both in separate thread pools which act differently. Some frameworks already do that for you out of the box. Nevertheless it’s important to be mindful of how distinct their usage is.
That’s precisely what I was talking about. I basically said it’s better to split an application by type of parallelism (CPU bound vs I/O bound) than to mix them.
An I/O heavy service benefits from having lots of available threads mapped to a smaller number of CPU cores, whereas a calculation heavy service benefits from pinning threads to cores to limit context switching. So scaling each will be quite different.
If you separate them into separate processes (one for I/O and one for compute), it’s much easier to scale them separately (more machines and whatnot). If I combine them, I’d need to continually balance how cores are split between concerns, and I wouldn’t have as much control over the types of cores (I/O is happy with lots of generic cores, whereas compute would benefit from specialized instructions).
So that’s my practical application of the “separate thread pools” idea, splitting thread pools at the process boundary is usually useful as an application grows in complexity. This increases latency, but it enables other types of tuning.
I don’t think your comment applies to the discussion. One of the thread pools mentioned is for IO-bound applications, which means things like sending HTTP requests.
Even if somehow you think it’s a good idea to move this class of tasks to a separate process, you will still have a very specific thread pool that can easily overcommit because most tasks end up idling while waiting for data to arrive.
The main take is that there are at least two classes of background tasks that have very distinct requirements and usage patterns. It’s important to handle both in separate thread pools which act differently. Some frameworks already do that for you out of the box. Nevertheless it’s important to be mindful of how distinct their usage is.
That’s precisely what I was talking about. I basically said it’s better to split an application by type of parallelism (CPU bound vs I/O bound) than to mix them.
An I/O heavy service benefits from having lots of available threads mapped to a smaller number of CPU cores, whereas a calculation heavy service benefits from pinning threads to cores to limit context switching. So scaling each will be quite different.
If you separate them into separate processes (one for I/O and one for compute), it’s much easier to scale them separately (more machines and whatnot). If I combine them, I’d need to continually balance how cores are split between concerns, and I wouldn’t have as much control over the types of cores (I/O is happy with lots of generic cores, whereas compute would benefit from specialized instructions).
So that’s my practical application of the “separate thread pools” idea, splitting thread pools at the process boundary is usually useful as an application grows in complexity. This increases latency, but it enables other types of tuning.