HPA Recommendation(Alpha)

Introduce for HPA Recommendation

Kubernetes’ users want to use HPA to optimize resource utilization. But it is often that we don’t know which applications are suitable for HPA or how to configure the parameters of HPA. With HPA Recommendation you can analyze the actual application usage and get recommended configurations. You can use it to improve application resource utilization.

HPA recommendation is still in Alpha phase, comments are welcome.


In Kubernetes, the HPA (HorizontalPodAutoscaler) automatically updates the workload replicas (such as Deployment or StatefulSet) to meet the target utilization. However, in the actual world, we observe following problems:

  • Some applications should improve resource utilization through HPA, but HPA is not configured
  • Some HPA configuration is not reasonable, can not effectively perform autoscaling, also can not improve resource utilization.

Based on the historical metrics data and algorithm analysis, HPA Recommendation provide the following suggestions: Which applications are suitable for HPA and How to configure it.


An HPA recommendation sample yaml looks like below:

apiVersion: analysis.crane.io/v1alpha1
kind: Recommendation
    analysis.crane.io/recommendation-rule-name: workload-hpa
    analysis.crane.io/recommendation-rule-recommender: HPA
    analysis.crane.io/recommendation-rule-uid: 0214c84b-8b39-499b-a7c6-559ac460695d
    analysis.crane.io/recommendation-target-kind: Rollout
    analysis.crane.io/recommendation-target-name: eshop
    analysis.crane.io/recommendation-target-version: v1alpha1
  name: workload-hpa-hpa-blr4r
  namespace: zytms
    - apiVersion: analysis.crane.io/v1alpha1
      blockOwnerDeletion: false
      controller: false
      kind: RecommendationRule
      name: workload-hpa
      uid: 0214c84b-8b39-499b-a7c6-559ac460695d
  adoptionType: StatusAndAnnotation
    completionStrategyType: Once
    apiVersion: argoproj.io/v1alpha1
    kind: Rollout
    name: eshop
    namespace: eshop
  type: HPA
  action: Create
  lastUpdateTime: "2022-12-05T06:12:54Z"
  recommendedInfo: '{"kind":"EffectiveHorizontalPodAutoscaler","apiVersion":"autoscaling.crane.io/v1alpha1","metadata":{"name":"eshop","namespace":"eshop","creationTimestamp":null},"spec":{"scaleTargetRef":{"kind":"Rollout","name":"eshop","apiVersion":"argoproj.io/v1alpha1"},"minReplicas":1,"maxReplicas":1,"scaleStrategy":"Preview","metrics":[{"type":"Resource","resource":{"name":"cpu","target":{"type":"Utilization","averageUtilization":58}}},{"type":"Pods","pods":{"metric":{"name":"k8s_pod_cpu_core_used"},"target":{"type":"AverageValue","averageValue":"500m"}}}]},"status":{}}'
  recommendedValue: |
      maxReplicas: 1
      - resource:
          name: cpu
            averageUtilization: 58
            type: Utilization
        type: Resource
      - pods:
            name: k8s_pod_cpu_core_used
            averageValue: 500m
            type: AverageValue
        type: Pods
      minReplicas: 1    

In this sample:

  • Recommendation TargetRef point to a ArgoRollout in eshop namespace: eshop
  • Recommendation type is HPA
  • adoptionType is StatusAndAnnotation,indicated that put recommendation result in recommendation.status and Workload’s Annotation
  • recommendedInfo shows the recommended HPA configuration.(recommendedValue is deprecated)
  • action is Create,If existing EHPA in k8s cluster, then the action will be Patch


The process for one HPA recommendation:

  1. Query the historical CPU and Memory usage of the Workload for the past week by monitoring system.
  2. Use DSP algorithm to predict the CPU usage in the future.
  3. Calculate the replicas for both CPU and memory, then choose a larger one.
  4. Calculate the historical CPU usage fluctuation and minimum usage, and filter out the suitable Workload for HPA
  5. Calculate the targetUtilization based on the peak CPU utilization of the pod
  6. Calculate the recommended maxReplicas based on the recommended targetUtilization
  7. Assemble the targetUtilization, maxReplicas, and minReplicas into a complete EHPA object as recommended result

How to filter the suitable workload suitable for HPA

It should meet the following conditions:

  1. The Workload is healthy. For example, most of the Pods are running
  2. There are peaks and troughs in CPU usage. A fixed number of replicas is recommended if the usage is largely smooth and steady or completely random
  3. Workload with a certain amount cpu usage. If the cpu usage is very low for a long time, HPA is no need even if there is some fluctuation

The following is a typical workload with peaks and troughs which is suitable for HPA.

Algorithm for recommend MinReplicas

The method is consistent with replica recommendation. Please referring to: Replicas Recommendation

Accepted resources

Support StatefulSet and Deployment by default,but all workloads that support Scale SubResource are supported.


Configuration itemsDefaultDescription
workload-min-replicas1Workload replicas that less than this value will abort recommendation
pod-min-ready-seconds30Defines the min seconds to identify Pod is ready
pod-available-ratio0.5Workload ready Pod ratio that less than this value will abort recommendation
default-min-replicas1default minReplicas
cpu-percentile0.95Percentile for historical cpu usage
mem-percentile0.95Percentile for historical memory usage
cpu-target-utilization0.5Target of CPU peak historical usage
mem-target-utilization0.5Target of Memory peak historical usage
predictablefalseWhen set to true, it will not recommend for HPA if CPU usage is not predictable
reference-hpatrueThe recommended result will inherits custom/external metric from existing ehpa
min-cpu-usage-threshold1Workload CPU peak usage that less than this value will abort recommendation
fluctuation-threshold1.5Workload CPU usage fluctuation that less than this value will abort recommendation
min-cpu-target-utilization30minimum CPU TargetUtilization
max-cpu-target-utilization75maximum CPU TargetUtilization
max-replicas-factor3the factor when calculate maxReplicas

How to update recommendation configuration please refer to:Recommendation Framework