HPA Autoscaling: Signals, Delays, and Traps

The Horizontal Pod Autoscaler (HPA) looks deceptively simple: set a target CPU percentage, and Kubernetes scales your pods automatically. In practice, teams discover that HPA reacts too slowly to traffic spikes, oscillates between scaling up and down, or scales on entirely the wrong signals. The gap between “HPA works” and “HPA works well for my traffic pattern” is where most autoscaling frustration lives.

I’ve watched this play out repeatedly. An e-commerce team configures HPA with a 50% CPU target. During a flash sale, traffic spikes 10x in 30 seconds. HPA takes 15 seconds to detect the load via the metrics server, another 15 seconds for the controller sync, then the stabilization window kicks in. Meanwhile, new pods need to be scheduled, images pulled, and readiness probes passed. By the time capacity catches up—3+ minutes later — frustrated users have already left. The service survived, but barely.

The lesson: HPA is reactive, not predictive. By the time it decides to scale, your workload is already under stress. The goal of HPA tuning is to minimize that reaction time while avoiding oscillation — a balance that requires understanding your traffic patterns, choosing the right metrics, and configuring stabilization windows that match your workload’s characteristics.

HPA Fundamentals

How HPA Works

The HPA controller runs a simple loop: every 15 seconds (by default), it queries the metrics server, calculates how many replicas are needed to hit the target utilization, and adjusts the deployment accordingly. The scaling formula is straightforward:

If you have 3 replicas running at 75% CPU with a target of 50%, HPA calculates replicas. The ceiling function ensures you always round up — better to have slightly more capacity than slightly less.

A basic HPA configuration targets CPU utilization across all pods. The scaleTargetRef points to the deployment you want to scale, and minReplicas/maxReplicas set the bounds:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server

  minReplicas: 3
  maxReplicas: 50

  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

The 50% target might seem conservative, but it’s intentional. You want headroom to absorb traffic increases while HPA scales up. A target of 80% means you’re already near capacity when HPA decides to act — and by the time new pods are ready, you’ve been overloaded for minutes.

The Delay Problem

The HPA loop sounds fast—15-second intervals — but the total time from “traffic spike begins” to “new capacity receives traffic” is much longer. Every step in the pipeline adds latency.

Here’s a best-case timeline for a traffic spike (assuming 15-second metrics resolution and cached images):

That’s 80+ seconds of degraded performance with aggressive tuning and favorable conditions. With default settings — which include a 300-second stabilization window for scale-up in older Kubernetes versions — it’s even worse.

Metric Selection

CPU vs Custom Metrics

The metric you choose determines how quickly HPA reacts — and whether it reacts to the right signal at all. Most teams start with CPU utilization because it’s built-in, but CPU is a lagging indicator: by the time CPU spikes, requests are already queuing. For web services, requests per second or queue depth are leading indicators that increase before your system shows stress.

Custom Metrics with Prometheus

Custom metrics let you scale on application-specific signals instead of generic resource utilization. The setup requires three components: your application exposing metrics, Prometheus scraping them, and an adapter translating Prometheus queries into the Kubernetes custom metrics API.

HPA supports three metric types, each suited to different scenarios:

Here’s an HPA configuration using all three types. HPA evaluates each metric independently and scales to the highest replica count any metric requests:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 50
  metrics:
    # Pod metric: requests per second
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"  # 100 RPS per pod

    # Pod metric: P95 latency
    - type: Pods
      pods:
        metric:
          name: http_request_duration_p95
        target:
          type: AverageValue
          averageValue: "200m"  # 200ms P95 target

    # External metric: queue depth
    - type: External
      external:
        metric:
          name: rabbitmq_queue_messages
          selector:
            matchLabels:
              queue: order-processing
        target:
          type: AverageValue
          averageValue: "50"  # 50 messages per pod

The Prometheus Adapter translates PromQL queries into the custom metrics API format. Each rule maps a Prometheus metric to a Kubernetes resource (namespace, pod) and defines how to query it:

apiVersion: v1
kind: ConfigMap
metadata:
  name: adapter-config
  namespace: custom-metrics
data:
  config.yaml: |
    rules:
      # Convert http_requests_total counter to per-second rate
      - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
        resources:
          overrides:
            namespace: {resource: "namespace"}
            pod: {resource: "pod"}
        name:
          matches: "^(.*)_total$"
          as: "${1}_per_second"
        metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'

      # Calculate P95 from histogram buckets
      - seriesQuery: 'http_request_duration_seconds_bucket{namespace!="",pod!=""}'
        resources:
          overrides:
            namespace: {resource: "namespace"}
            pod: {resource: "pod"}
        name:
          as: "http_request_duration_p95"
        metricsQuery: 'histogram_quantile(0.95, sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (le, <<.GroupBy>>))'

The [2m] window in the PromQL queries smooths out brief spikes. A shorter window (30 seconds) makes scaling more responsive but increases oscillation risk. A longer window (5 minutes) stabilizes scaling but delays reaction to genuine load changes. Two minutes is a reasonable starting point.

HPA Behavior Configuration

Stabilization and Scaling Policies

The behavior field in HPA v2 gives you fine-grained control over scaling speed. Two mechanisms work together: stabilization windows prevent thrashing by requiring metrics to stay above/below thresholds for a duration before acting, and scaling policies limit how many replicas can be added or removed per time period.

The key insight is that scaling up and scaling down should be asymmetric. When traffic spikes, you want capacity now — waiting costs customer experience. When traffic drops, you want to wait — scaling down too fast means you’ll scale right back up if traffic returns, wasting the pods you just terminated.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 50
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0        # Scale immediately
      policies:
        - type: Percent
          value: 100                        # Double capacity
          periodSeconds: 15
        - type: Pods
          value: 4                          # Or add 4 pods
          periodSeconds: 15
      selectPolicy: Max                     # Use whichever adds more

    scaleDown:
      stabilizationWindowSeconds: 300       # Wait 5 minutes
      policies:
        - type: Percent
          value: 10                         # Remove 10% of pods
          periodSeconds: 60
        - type: Pods
          value: 2                          # Or remove 2 pods
          periodSeconds: 60
      selectPolicy: Min                     # Use whichever removes fewer

The selectPolicy field determines how HPA chooses between multiple policies:

Common Behavior Patterns

Different traffic patterns call for different behavior configurations. The tradeoff is always between responsiveness (scale fast to handle spikes) and stability (avoid oscillation and wasted resources).

For flash sales or viral traffic, you want the most aggressive scale-up possible. Zero stabilization means HPA acts on the first evaluation that shows high load. A 200% policy means you can double capacity every 15 seconds — going from 5 to 10 to 20 to 40 pods in under a minute if needed:

behavior:
  scaleUp:
    stabilizationWindowSeconds: 0
    policies:
      - type: Percent
        value: 200
        periodSeconds: 15
    selectPolicy: Max
  scaleDown:
    stabilizationWindowSeconds: 300
    policies:
      - type: Percent
        value: 10
        periodSeconds: 60
    selectPolicy: Min

For predictable business-hours traffic, you can afford slower scaling. A 60-second stabilization window filters out brief metric noise, and a 50% scale-up rate is fast enough to handle gradual morning ramp-up without over-provisioning:

behavior:
  scaleUp:
    stabilizationWindowSeconds: 60
    policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    selectPolicy: Max
  scaleDown:
    stabilizationWindowSeconds: 600
    policies:
      - type: Percent
        value: 20
        periodSeconds: 120
    selectPolicy: Min

Tuning for Traffic Patterns

The “right” HPA configuration depends entirely on your traffic. An e-commerce site with predictable business-hours traffic needs different settings than a social platform that might go viral at any moment. Generic configurations either scale too slowly for spiky traffic or waste money on stable traffic. Understanding your traffic patterns lets you tune HPA for your specific tradeoffs.

Traffic Pattern Analysis

Before tuning HPA, you need to understand your traffic. Four characteristics matter most:

These characteristics map to specific HPA strategies. The table below shows common traffic patterns and the scaling approaches that work best for each:

The key insight is that HPA alone can’t handle instantaneous spikes. If traffic can increase 10x in 10 seconds, you need capacity already running when the spike hits. HPA’s role becomes handling the variance around your pre-provisioned baseline, not catching up from zero.

Combining HPA with Scheduled Scaling

For predictable traffic patterns, combine HPA with scheduled scaling. KEDA (Kubernetes Event-Driven Autoscaling) supports cron triggers that set minimum replica counts at specific times, while HPA handles real-time adjustments within those bounds.

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-server-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: api-server
  minReplicaCount: 3
  maxReplicaCount: 50
  triggers:
    # Baseline: CPU-based reactive scaling
    - type: cpu
      metadata:
        type: Utilization
        value: "50"

    # Pre-scale for business hours (8 AM Mon-Fri)
    - type: cron
      metadata:
        timezone: America/New_York
        start: "0 8 * * 1-5"
        end: "0 9 * * 1-5"
        desiredReplicas: "10"

    # Pre-scale for known flash sale (15th of month)
    - type: cron
      metadata:
        timezone: America/New_York
        start: "0 11 15 * *"
        end: "0 15 15 * *"
        desiredReplicas: "30"

If you don’t have KEDA, a simple CronJob can pre-scale deployments before known traffic events:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: prescale-for-business-hours
  namespace: production
spec:
  schedule: "0 7 * * 1-5"  # 7 AM Mon-Fri, 1 hour before peak
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: scaler-sa  # Needs patch permissions
          containers:
            - name: scaler
              image: bitnami/kubectl:latest
              command: ["kubectl", "scale", "deployment/api-server",
                        "--replicas=10", "-n", "production"]
          restartPolicy: OnFailure

The pattern is simple: pre-scale 30-60 minutes before expected traffic, then let HPA handle the actual load. If your prediction was low, HPA scales up. If your prediction was high, HPA (eventually) scales down. Either way, you’re not scrambling to add capacity while users wait.

Debugging HPA Issues

HPA failures are frustrating because they’re often silent — you don’t notice until traffic spikes and capacity doesn’t follow. The good news is that HPA exposes its decision-making through kubectl describe hpa, and most problems fall into a few categories: metrics unavailable, scaling too slow, oscillating, or not scaling down.

Common Problems and Solutions

Problem: HPA stuck at minReplicas despite high load

This usually means HPA can’t see your metrics. Check the Conditions section in kubectl describe hpa—look for “unable to get metrics” or “AbleToScale: False”. The most common causes:

# Check if metrics are available
kubectl top pods -n production

# Check metrics server health
kubectl get apiservices | grep metrics

# Verify pods have resource requests
kubectl get pod <pod-name> -o jsonpath='{.spec.containers[*].resources.requests}'

Problem: HPA oscillates rapidly between replica counts

Thrashing — scaling up, then immediately scaling down, then up again — indicates either too-short stabilization windows or too-aggressive targets. Watch the replica count over time:

# Watch HPA changes in real-time
kubectl get hpa api-server-hpa -n production -w

# Check recent scaling events
kubectl describe hpa api-server-hpa -n production | grep -A 20 "Events:"

Fixes: Increase stabilizationWindowSeconds (especially for scale-down), lower the target utilization (50% instead of 80%), or smooth your metrics with longer averaging windows in your Prometheus queries.

Problem: HPA scales too slowly for traffic spikes

If HPA is working but not fast enough, check the behavior configuration. Default stabilization windows (especially the 300-second scale-up window in older Kubernetes versions) add significant delay.

# Check current behavior configuration
kubectl get hpa api-server-hpa -n production -o yaml | grep -A 30 "behavior:"

# Check last scale time
kubectl describe hpa api-server-hpa -n production | grep "Last Scale Time"

Fixes: Set scaleUp.stabilizationWindowSeconds: 0, increase scale-up policy percentages, optimize pod startup time, or raise minReplicas to maintain headroom.

Problem: HPA never scales down

Pods sitting idle waste money. If HPA won’t scale down even when load is low, check whether scale-down is accidentally disabled or the stabilization window is extremely long.

# Check scale-down policies
kubectl get hpa api-server-hpa -n production -o jsonpath='{.spec.behavior.scaleDown}' | jq .

# Verify metrics are actually below target
kubectl get hpa api-server-hpa -n production

Look for selectPolicy: Disabled in scale-down, or a policy with value: 0 that prevents any pod removal.

Debugging Decision Tree

When HPA isn’t behaving as expected, work through this diagnostic flow:

Essential Debugging Commands

These commands cover most HPA debugging scenarios:

#!/bin/bash

HPA="api-server-hpa"
NS="production"

# Quick status check
kubectl get hpa $HPA -n $NS

# Full details including conditions and events
kubectl describe hpa $HPA -n $NS

# Current metrics in JSON format
kubectl get hpa $HPA -n $NS -o jsonpath='{.status.currentMetrics}' | jq .

# Scaling history from events
kubectl get events -n $NS \
  --field-selector involvedObject.name=$HPA \
  --sort-by='.lastTimestamp' | tail -15

# Current pod resource usage
kubectl top pods -n $NS -l app=api-server

Advanced Patterns

Standard HPA covers most use cases, but some scenarios require more sophisticated approaches: combining multiple metrics so any bottleneck triggers scaling, scaling to zero for idle workloads, or handling event-driven architectures where traditional metrics don’t apply.

Multi-Metric HPA

HPA can evaluate multiple metrics simultaneously. When you configure several metrics, HPA calculates the desired replica count for each and uses the maximum — the metric requiring the most replicas wins. This ensures you scale up when any resource becomes constrained, not just one.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-server-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-server
  minReplicas: 3
  maxReplicas: 50
  metrics:
    # CPU: catches compute-bound bottlenecks
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50

    # RPS: catches request-heavy scenarios before CPU spikes
    - type: Pods
      pods:
        metric:
          name: http_requests_per_second
        target:
          type: AverageValue
          averageValue: "100"

    # Latency: catches degraded performance regardless of cause
    - type: Pods
      pods:
        metric:
          name: http_request_duration_p99_seconds
        target:
          type: AverageValue
          averageValue: "500m"  # Scale if P99 > 500ms

Here’s how the calculation works. Suppose current state is 5 replicas:

HPA takes the maximum: 6 replicas. The latency metric calculates fewer replicas than currently running (4 < 5), which would suggest scaling down if it were the only metric — but CPU and RPS both want more capacity, so we scale up. This “max wins” behavior ensures you don’t under-provision when any metric indicates stress.

This pattern is particularly useful for services with mixed workloads — some requests are CPU-heavy, others are I/O-heavy. A single metric might miss bottlenecks that only affect certain request types.

KEDA for Event-Driven Scaling

KEDA (Kubernetes Event-Driven Autoscaling) extends HPA with two capabilities standard HPA lacks: scaling to zero, and scaling on external event sources like message queues, databases, or cloud services.

For queue-based workers, KEDA scales based on queue depth rather than pod metrics. When the queue is empty, KEDA can scale to zero pods, saving resources. When messages arrive, KEDA spins up pods to process them:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: order-processor-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: order-processor
  minReplicaCount: 1
  maxReplicaCount: 100
  idleReplicaCount: 0           # Scale to zero when idle
  cooldownPeriod: 300           # Wait 5 min before scaling to zero
  pollingInterval: 15           # Check queue every 15s
  triggers:
    - type: rabbitmq
      metadata:
        host: amqp://rabbitmq.messaging:5672
        queueName: orders
        mode: QueueLength
        value: "10"             # Target 10 messages per pod
    - type: rabbitmq
      metadata:
        host: amqp://rabbitmq.messaging:5672
        queueName: orders
        mode: MessageRate
        value: "50"             # Also consider message arrival rate

KEDA also supports Prometheus as a trigger source, letting you scale on any metric Prometheus collects — including business metrics like active users or transaction volume:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: api-scaler
  namespace: production
spec:
  scaleTargetRef:
    name: api-server
  minReplicaCount: 3
  maxReplicaCount: 50
  triggers:
    - type: prometheus
      metadata:
        serverAddress: http://prometheus.monitoring:9090
        threshold: "100"
        query: sum(rate(http_requests_total{deployment="api-server"}[2m]))

The key difference from standard HPA with Prometheus Adapter is KEDA’s native support for scale-to-zero and its simpler configuration for external sources. If you need either capability, KEDA is the standard solution.

Conclusion

HPA tuning comes down to four principles:

The tuning process is iterative. Start with defaults, run under real load, and watch what happens. If you’re scaling too slowly, reduce stabilization windows and increase policy percentages. If you’re oscillating, increase stabilization and lower your target utilization. If you’re wasting money on idle pods, tighten scale-down policies or raise target utilization. There’s no universal “best” configuration — only the configuration that matches your traffic pattern.