Skip to main content

Capacity Planning

1 - Introduction​

1.1 Definition​

Capacity Planning = The process of determining the resources needed to meet future demand.

1.2 Why it matters​

ScenarioConsequence
Under-provisionedOutages, latency, customer loss
Over-provisionedWasted resources, high costs
Well plannedOptimal performance, controlled costs

2 - Capacity metrics​

2.1 Resources to monitor​

capacity_metrics:
  compute:
    - CPU utilization
    - Memory usage
    - Network bandwidth
    
  storage:
    - Disk space
    - IOPS
    - Throughput
    
  application:
    - Concurrent connections
    - Requests per second
    - Queue depth
    
  database:
    - Connection pool
    - Query throughput
    - Replication lag

2.2 Saturation Points​

saturation_thresholds:
  cpu:
    warning: 70%
    critical: 85%
    action: "Scale up or out"
    
  memory:
    warning: 75%
    critical: 90%
    action: "Add memory or instances"
    
  disk:
    warning: 70%
    critical: 85%
    action: "Extend storage"
    
  connections:
    warning: 70%
    critical: 85%
    action: "Increase pool or scale"

3 - Demand Forecasting​

3.1 Methods​

forecasting_methods:
  historical_trend:
    description: "Extrapolation of past data"
    best_for: "Stable organic growth"
    
  seasonal:
    description: "Recurring patterns (daily, weekly, yearly)"
    best_for: "E-commerce, B2C"
    
  event_driven:
    description: "Planned events (promos, launches)"
    best_for: "Marketing campaigns"
    
  regression:
    description: "Statistical model"
    best_for: "Complex predictions"

3.2 Prometheus Prediction​

# Linear prediction: disk space in 7 days
predict_linear(node_filesystem_avail_bytes[7d], 7*24*3600)

# Prediction: when the disk will be full
(
  node_filesystem_avail_bytes
  /
  deriv(node_filesystem_avail_bytes[7d])
) / 3600 / 24  # Days remaining

3.3 Growth Modeling​

growth_model:
  current_metrics:
    users: 100000
    rps: 1000
    instances: 10
    
  growth_rate:
    monthly: 10%
    yearly: 214%  # Compound
    
  projection_6_months:
    users: 177000
    rps: 1770
    instances_needed: 18

4 - Load Testing​

4.1 Types of tests​

TypeObjectiveDuration
SmokeBasic validationMinutes
LoadNormal performanceHours
StressFind the limitsHours
SpikeSudden peaksMinutes
SoakEnduranceDays

4.2 k6 Load Test​

// load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '5m', target: 100 },  // Ramp up
    { duration: '30m', target: 100 }, // Steady
    { duration: '5m', target: 200 },  // Peak
    { duration: '5m', target: 0 },    // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(99)<500'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('https://api.example.com/endpoint');
  check(res, {
    'status is 200': (r) => r.status === 200,
    'response time < 500ms': (r) => r.timings.duration < 500,
  });
  sleep(1);
}

4.3 Determining the limits​

capacity_limits:
  test_results:
    max_rps: 5000
    breaking_point_rps: 6000
    p99_at_max: 450ms
    error_rate_at_max: 0.5%
    
  headroom:
    target: 30%  # Safety headroom
    safe_max_rps: 3500
    
  scaling_trigger:
    cpu: 70%
    rps_per_instance: 350

5 - Capacity Planning Process​

5.1 Annual Planning​

annual_planning:
  q1_review:
    - Review past year growth
    - Analyze seasonal patterns
    - Update growth projections
    
  q2_planning:
    - Define capacity requirements
    - Budget allocation
    - Infrastructure roadmap
    
  q3_implementation:
    - Provision resources
    - Optimize existing systems
    - Load testing
    
  q4_preparation:
    - Holiday/peak preparation
    - Buffer provisioning
    - Incident readiness

5.2 Capacity Model​

capacity_model:
  service: api-gateway
  
  resource_per_unit:
    cpu_cores: 0.5
    memory_gb: 1
    rps_capacity: 100
    
  current_state:
    instances: 20
    total_capacity: 2000 rps
    current_usage: 1400 rps
    utilization: 70%
    
  growth_projection:
    expected_growth: 50%
    required_capacity: 2100 rps
    required_instances: 21
    recommended: 25  # With 20% buffer

6 - Auto-scaling​

6.1 Kubernetes HPA​

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 50
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 80
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 10
          periodSeconds: 60

6.2 Custom Metrics​

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
  metrics:
    - type: External
      external:
        metric:
          name: queue_depth
          selector:
            matchLabels:
              queue: orders
        target:
          type: AverageValue
          averageValue: "100"

7 - Cost Optimization​

7.1 Right-sizing​

rightsizing:
  analysis:
    - Review actual resource usage
    - Compare to requested
    - Identify over-provisioned
    
  actions:
    - Reduce oversized instances
    - Use appropriate instance types
    - Implement resource quotas

7.2 Reserved vs On-demand​

capacity_strategy:
  reserved:
    for: "Baseline capacity"
    percentage: 60%
    savings: "30-60%"
    
  on_demand:
    for: "Variable capacity"
    percentage: 30%
    
  spot:
    for: "Non-critical, interruptible"
    percentage: 10%
    savings: "60-90%"

Summary​

In this chapter, we learned:

  • The capacity metrics
  • Demand Forecasting
  • Load Testing (k6)
  • The Capacity Planning process
  • Kubernetes Auto-scaling
  • Cost optimization

Next step​

In the next chapter, we will look at Release Engineering.

→ Next chapter: Release Engineering

← Back to table of contents