Supply and Demand: The Two Failure Modes
| State | Definition | Symptoms | Consequences | Root Cause |
|---|---|---|---|---|
| Undercapacity | Demand exceeds available supply | Slow response times, queued requests, timeouts, error pages | Outages, lost revenue, SLA penalties, customer churn, reputational damage | Underestimated growth, unexpected demand spike, failure to act on monitoring data |
| Overcapacity | Supply far exceeds actual demand | Low utilization metrics (5–10% CPU), idle servers, unused licenses | Financial waste: idle hardware costs money; idle cloud instances generate bills; over-hired staff exceeds value produced | Over-provisioning "for safety," over-forecasting growth, failure to right-size after demand drops |
| Correct Capacity | Supply matches demand with appropriate headroom | Normal response times, utilization in healthy range (40–70%), no queuing | Good performance, cost efficiency, no outages under normal conditions | Active monitoring, accurate forecasting, regular right-sizing reviews |
The Three Dimensions of Capacity Planning
| Dimension | What It Covers | Speed to Scale Up | Speed to Scale Down | Key Strategies |
|---|---|---|---|---|
| People | Human staff required to deliver services — call center agents, analysts, admins, technicians | Weeks to months (recruit → interview → hire → onboard → train → productive) | Weeks to months (redeployment, notice periods, severance); morale and legal costs | Contract staffing, cross-training, outsourcing, remote work, longer forecast horizons |
| Technology | Software architectures and services that must support growing/shrinking load | Fast if designed for it: add web servers in minutes; database sharding requires planning | Fast: remove servers, reduce instances, no financial/legal friction | Load balancers + multiple web servers; database clustering and sharding; cloud on-demand services |
| Infrastructure | Underlying compute, storage, network hardware and cloud resources everything runs on | Physical: weeks/months; Cloud: minutes | Physical: slow (hardware persists); Cloud: fast (terminate instances) | Cloud-first for flexibility; physical for predictable steady-state workloads; hybrid for both |
Physical Infrastructure vs. Cloud Infrastructure
| Attribute | Physical Infrastructure | Cloud Infrastructure |
|---|---|---|
| Expense Type | Capital expenditure (CapEx) — large upfront purchase, depreciates | Operational expenditure (OpEx) — pay-per-use, no depreciation |
| Provisioning Time | Weeks to months: order → ship → rack → configure → test → deploy | Minutes: web console click or API call; no hardware to receive |
| Scaling Speed | Slow — must order, wait for delivery, install; downtime may be required | Fast — add/remove instances in real time; auto-scaling available |
| Cost When Idle | Continues to consume power, cooling, rack space; cost does not decrease with utilization | Pay only when running (or scaled to zero); idle instances can be terminated |
| Unexpected Spikes | Cannot respond quickly; must have pre-purchased headroom or accept degraded performance | Provision additional instances immediately; serve the spike; decommission when done |
| Best For | Predictable, steady-state workloads with known capacity requirements; compliance/latency requirements | Variable workloads, unpredictable demand, new deployments with uncertain usage patterns |
Vertical Scaling vs. Horizontal Scaling
| Attribute | Vertical Scaling (Scale Up) | Horizontal Scaling (Scale Out) |
|---|---|---|
| Method | Add more CPU, RAM, storage, or bandwidth to one existing server | Add more servers behind a load balancer; distribute workload across all |
| Architectural Change | None — existing application runs on a bigger server unchanged | Required — application must run as multiple independent instances simultaneously |
| Ceiling | Yes — maximum CPU count, maximum RAM, fastest storage; hard ceiling exists | No practical ceiling — keep adding servers as demand grows (financial limit only) |
| Redundancy | None — one bigger server is still one server; single point of failure | Built-in — if one server fails, others continue; load balancer routes around failures |
| Downtime to Scale | Physical: yes (must shut down to add RAM); virtual: depends on platform | No — add servers to pool without interrupting existing servers |
| Best For | Stateful workloads, legacy apps, databases that can't be distributed, quick short-term capacity boost | Stateless web/app servers, microservices, modern cloud architectures, workloads with long-term growth |
| Dominant Use Case | Older on-premises systems; first line of defense for constrained databases | Web servers, application tiers, cloud auto-scaling, modern distributed systems |
Technology Scalability by Service Type
| Service Type | Scaling Approach | Key Component | How It Works | Complexity |
|---|---|---|---|---|
| Web Services | Horizontal (scale out) | Load balancer | Load balancer routes incoming requests across multiple web server instances; add/remove servers from pool with no user impact | Low — stateless web servers are easy to add |
| Database — Clustering | Horizontal (shared work) | Cluster manager | Multiple SQL servers work together; query load distributed across cluster members; replication maintains consistency | Medium — consistency across cluster requires coordination |
| Database — Sharding | Horizontal (data split) | Shard routing logic | Database divided into shards across servers (A–M on server 1, N–Z on server 2); each server handles a subset of data; reduces per-server load | High — cross-shard queries require application-layer joins; best designed from start |
| Cloud Services | Elastic (auto-scale) | Auto-scaling group | Cloud platform monitors metrics; automatically provisions additional instances when demand rises; terminates idle instances when demand falls | Low to set up; medium to configure correctly; handles scaling automatically |
Fixed Capacity vs. Elasticity: The Provisioning Dilemma
| Strategy | Provisioned At | During Normal Demand | During Peak Demand | Financial Outcome |
|---|---|---|---|---|
| Provision for Peak | Maximum expected demand | Most resources idle; paying full cost for unused capacity | Sufficient — handles the peak with no degradation | High cost; significant waste during off-peak periods |
| Provision for Normal | Average/normal demand | Efficient — resources well utilized, minimal waste | Insufficient — system overwhelmed; slowdowns and outages | Low normal cost; high incident cost when peaks occur |
| Elasticity (Auto-Scale) | Minimum baseline; auto-scales to peak as needed | Efficient — baseline resources only; minimal waste | Scales out automatically; serves demand; scales back in when peak ends | Optimal — pays for actual demand in real time; no idle waste; no peak outages |
Monitoring and Forecasting: Proactive vs. Reactive
| Approach | Trigger | Action Timing | Result | Risk |
|---|---|---|---|---|
| Reactive Capacity Management | Outage or severe degradation occurs | After the problem is already impacting users | Emergency procurement (expensive), service disruption during response, loss of trust | High — cannot respond faster than the procurement/provisioning timeline |
| Monitoring-Driven | Utilization metric approaches threshold (e.g., CPU at 75% and rising) | Before the limit is reached; warning while headroom still exists | Planned expansion with time to procure, configure, and test before needed | Low — acts on early warning signals before users are affected |
| Forecast-Driven | Trend analysis, seasonal knowledge, or business growth projections | Weeks or months in advance of the expected need | Capacity ready before demand arrives; no rush, no emergency costs | Lowest — capacity available ahead of need; even physical infrastructure lead times are covered |