What is a typical trigger for horizontal scaling of application servers?

What is a typical trigger for horizontal scaling of application servers?

• A. CPU utilization above 90% is the typical trigger for horizontal scaling.
• B. CPU utilization above 70-80% is the typical trigger for horizontal scaling, not memory or connections.
• C. Scale only when memory usage exceeds 95%.
• D. Scale only when requests per second double.

Answer: (B)

Horizontal scaling is driven by how hard the servers are working, and CPU utilization around 70–80% is a common trigger to add more instances. This range keeps the system with enough headroom to handle bursts without letting latency spike, while still using capacity efficiently. CPU serves as a practical proxy for processing load, so when it climbs toward that range, adding instances helps keep response times stable. Relying on memory alone can be misleading since memory pressure doesn’t always translate to slower responses—memory can be used for caching, and its usage patterns don’t map directly to user-perceived performance. Connections fluctuating or skyrocketing don’t necessarily indicate a need for more servers in a straightforward way, since they depend on concurrency and back-end behavior. Likewise, scaling based solely on requests per second doesn’t account for the cost-to-performance balance; two workloads with the same RPS can have very different latencies. So the typical, best-aligned trigger is CPU utilization in the 70–80% range, balancing readiness to scale with efficient use of resources.

Horizontal scaling is driven by how hard the servers are working, and CPU utilization around 70–80% is a common trigger to add more instances. This range keeps the system with enough headroom to handle bursts without letting latency spike, while still using capacity efficiently. CPU serves as a practical proxy for processing load, so when it climbs toward that range, adding instances helps keep response times stable.

Relying on memory alone can be misleading since memory pressure doesn’t always translate to slower responses—memory can be used for caching, and its usage patterns don’t map directly to user-perceived performance. Connections fluctuating or skyrocketing don’t necessarily indicate a need for more servers in a straightforward way, since they depend on concurrency and back-end behavior. Likewise, scaling based solely on requests per second doesn’t account for the cost-to-performance balance; two workloads with the same RPS can have very different latencies.

So the typical, best-aligned trigger is CPU utilization in the 70–80% range, balancing readiness to scale with efficient use of resources.