Autoscaling, a key feature of Kubernetes, lets you improve the resource utilization of your cluster by automatically adjusting the application’s resources or replicas depending on the load at that time.

This blog talks about Pod Autoscaling in Kubernetes and how to set up and configure autoscalers to optimize the resource utilization of your application.

Horizontal Pod Autoscaling

What is the Horizontal Pod Autoscaler?

The Horizontal Pod Autoscaler (HPA) scales the number of pods of a replica-set/ deployment/ statefulset based on per-pod metrics received from resource metrics API (metrics.k8s.io) provided by metrics-server, the custom metrics API (custom.metrics.k8s.io), or the external metrics API (external.metrics.k8s.io).

‍Prerequisite

Verify that the metrics-server is already deployed and running using the command below, or deploy it using instructions here.

CODE: https://gist.github.com/velotiotech/199ac1a38a3ab0e3a07a043ef1daecca.js

HPA using Multiple Resource Metrics‍

HPA fetches per-pod resource metrics (like CPU, memory) from the resource metrics API and calculates the current metric value based on the mean values of all targeted pods. It compares the current metric value with the target metric value specified in the HPA spec and produces a ratio used to scale the number of desired replicas.

A. Setup: Create a Deployment and HPA resource

In this blog post, I have used the config below to create a deployment of 3 replicas, with some memory load defined by “--vm-bytes", "850M”.

CODE: https://gist.github.com/velotiotech/6a03575aa9153f04efe764133f3f4a6b.js

NOTE: It’s recommended not to use HPA and VPA on the same pods or deployments.

CODE: https://gist.github.com/velotiotech/76948daf9c35a4744d78972a1293aa3f.js

Lets create an HPA resource for this deployment with multiple metric blocks defined. The HPA will consider each metric one-by-one and calculate the desired replica counts based on each of the metrics, and then select the one with the highest replica count.

CODE: https://gist.github.com/velotiotech/b2987ff6717591d1925ffbfce925d3e8.js

• We have defined the minimum number  of replicas HPA can scale down to as 1 and the maximum number that it can scale up to as 10.
• Target Average Utilization and Target Average Values implies that the HPA should scale the replicas up/down to keep the Current Metric Value equal or closest to Target Metric Value.
  

B. Understanding the HPA Algorithm

CODE: https://gist.github.com/velotiotech/b592175a41a52bf0cdc44d6b1ff1b79a.js

• HPA calculates pod utilization as total usage of all containers in the pod divided by total request. It looks at all containers individually and returns if container doesn't have request.
• The calculated  Current Metric Value for memory, i,e., 894188202666m, is higher than the Target Average Value of 500Mi, so the replicas need to be scaled up.
• The calculated  Current Metric Value for CPU i.e., 36%, is lower than the Target Average Utilization of 50, so  hence the replicas need to be scaled down.
• Replicas are calculated based on both metrics and the highest replica count selected. So, the replicas are scaled up to 6 in this case.
  

HPA using Custom metrics

We will use the prometheus-adapter resource to expose custom application metrics to custom.metrics.k8s.io/v1beta1, which are retrieved by HPA. By defining our own metrics through the adapter’s configuration, we can let HPA perform scaling based on our custom metrics.

A. Setup: Install Prometheus Adapter

Create prometheus-adapter.yaml with the content below:

CODE: https://gist.github.com/velotiotech/7323ef8969bfeb40929b0d1205518642.js

CODE: https://gist.github.com/velotiotech/acffd596f1992431c283699aa3292f23.js

Once the charts are deployed, verify the metrics are exposed at v1beta1.custom.metrics.k8s.io:

CODE: https://gist.github.com/velotiotech/5b4297bc2d38b4d993d82541ec6f2685.js

You can see the metrics value of all the replicas in the output.

B. Understanding Prometheus Adapter Configuration

The adapter considers metrics defined with the parameters below:

1. seriesQuery tells the Prometheus Metric name to the adapter‍

2. resources tells which Kubernetes resources each metric is associated with or which labels does the metric include, e.g., namespace, pod etc.

3. metricsQuery is the actual Prometheus query that needs to be performed to calculate the actual values.

4. name with which the metric should be exposed to the custom metrics API

For instance, if we want to calculate the rate of container_network_receive_packets_total, we will need to write this query in Prometheus UI:

sum(rate(container_network_receive_packets_total{namespace="autoscale-tester",pod=~"autoscale-tester.*"}[10m])) by (pod)

This query is represented as below in the adapter configuration:

metricsQuery: 'sum(rate(<<.series>>{<<.labelmatchers>>}10m])) by (<<.groupby>>)'</.groupby></.labelmatchers></.series>

C. Create an HPA resource

Now, let's create an HPA resource with the pod metric packets_in using the config below, and then describe the HPA resource.

CODE: https://gist.github.com/velotiotech/06bc056955cf524784e006b8cca0efff.js

CODE: https://gist.github.com/velotiotech/c7782cc7e2c3dbe9f31bd896d5232843.js

Here, the current calculated metric value is 18666m. The m represents milli-units. So, for example, 18666m means 18.666 which is what we expect ((33 + 11 + 10 )/3 = 18.666). Since it's less than the target average value (i.e., 50), the HPA scales down the replicas to make the Current Metric Value : Target Metric Value ratio closest to 1. Hence, replicas are scaled down to 2 and later to 1.

‍Vertical Pod Autoscaling

What is Vertical Pod Autoscaler?

Vertical Pod autoscaling (VPA) ensures that a container’s resources are not under- or over-utilized. It recommends optimized CPU and memory requests/limits values, and can also automatically update them for you so that the cluster resources are efficiently used.

Architecture

VPA consists of 3 components:

• VPA admission controller
  Once you deploy and enable the Vertical Pod Autoscaler in your cluster, every pod submitted to the cluster goes through this webhook, which checks whether a VPA object is referencing it.

• VPA recommender
  The recommender pulls the current and past resource consumption (CPU and memory) data for each container from metrics-server running in the cluster and provides optimal resource recommendations based on it, so that a container uses only what it needs.

• VPA updater
  The updater checks at regular intervals if a pod is running within the recommended range. Otherwise, it accepts it for update, and the pod is evicted by the VPA updater to apply resource recommendation.
  

Installation

If you are on Google Cloud Platform, you can simply enable vertical-pod-autoscaling:‍

CODE: https://gist.github.com/velotiotech/65622276e2f9741e27231c3f72154fe2.js

To install it manually follow below steps:

• Verify that the metrics-server deployment is running, or deploy it using instructions here.

CODE: https://gist.github.com/velotiotech/2b38b711e5e35802168443b447ce5900.js

• Also, verify the API below is enabled:

CODE: https://gist.github.com/velotiotech/7609b41e20e60231aac6c2492bbfb44d.js

• Clone the kubernetes/autoscaler GitHub repository, and then deploy the Vertical Pod Autoscaler with the following command.

CODE: https://gist.github.com/velotiotech/d6fb0be829e747310de14f0324e6cf4b.js

Verify that the Vertical Pod Autoscaler pods are up and running:

CODE: https://gist.github.com/velotiotech/a1c1ca0d5a3a8536938a7c35af219ab1.js

VPA using Resource Metrics

A. Setup: Create a Deployment and VPA resource

Use the same deployment config to create a new deployment with "--vm-bytes", "850M". Then create a VPA resource in Recommendation Mode with updateMode : Off

CODE: https://gist.github.com/velotiotech/99d9368377626c8491040a02e6ce003b.js

• minAllowed is an optional parameter that specifies the minimum CPU request and memory request allowed for the container. 

• maxAllowed is an optional parameter that specifies the maximum CPU request and memory request allowed for the container.
  

B. Check the Pod’s Resource Utilization

Check the resource utilization of the pods. Below, you can see only ~50 Mi memory is being used out of 1000Mi and only ~30m CPU out of 1000m. This clearly indicates that the pod resources are underutilized.

CODE: https://gist.github.com/velotiotech/31f544044febaf3558794ca0bbb7287e.js

If you describe the VPA resource, you can see the Recommendations provided. (It may take some time to show them.)

CODE: https://gist.github.com/velotiotech/cdf95cfe43e033d32df25578466104b5.js

C. Understand the VPA recommendations

Target: The recommended CPU request and memory request for the container that will be applied to the pod by VPA.

Uncapped Target: The recommended CPU request and memory request for the container if you didn’t configure upper/lower limits in the VPA definition. These values will not be applied to the pod. They’re used only as a status indication.

Lower Bound: The minimum recommended CPU request and memory request for the container. There is a --pod-recommendation-min-memory-mb flag that determines the minimum amount of memory the recommender will set—it defaults to 250MiB.

Upper Bound: The maximum recommended CPU request and memory request for the container.  It helps the VPA updater avoid eviction of pods that are close to the recommended target values. Eventually, the Upper Bound is expected to reach close to target recommendation.

CODE: https://gist.github.com/velotiotech/d564f85bb52a39545d80326663eff416.js

D. VPA processing with Update Mode Off/Auto

Now, if you check the logs of vpa-updater, you can see it's not processing VPA objects as the Update Mode is set as Off.

CODE: https://gist.github.com/velotiotech/afdf359d54ac22c94c283cbc72f294a5.js

VPA allows various Update Modes, detailed here.

Let's change the VPA updateMode to “Auto” to see the processing.

As soon as you do that, you can see vpa-updater has started processing objects, and it's terminating all 3 pods.

CODE: https://gist.github.com/velotiotech/97e8491ea7eded7230225404c404cfec.js

You can also check the logs of vpa-admission-controller:

CODE: https://gist.github.com/velotiotech/2683d40c5449c38b8f7fa36dacf415d5.js

NOTE: Ensure that you have more than 1 running replicas. Otherwise, the pods won’t be restarted, and vpa-updater will give you this warning:

CODE: https://gist.github.com/velotiotech/c329dda66ae95e1ef259c36e2d9ef097.js

Now, describe the new pods created and check that the resources match the Target recommendations:

CODE: https://gist.github.com/velotiotech/a05c5adcd25bed54baef5e3196439d26.js

The Target Recommendation can not go below the minAllowed defined in the VPA spec.

‍E. Stress Loading Pods

Let’s recreate the deployment with memory request and limit set to 2000Mi and "--vm-bytes", "500M".

Gradually stress load one of these pods to increase its memory utilization.
You can login to the pod and run stress --vm 1 --vm-bytes 1400M --timeout 120000s.

CODE: https://gist.github.com/velotiotech/82c62a92355455571f3d82e3ccf1ca1f.js

You will notice that the VPA recommendation is also calculated accordingly and applied to all replicas.

CODE: https://gist.github.com/velotiotech/4f99a9a8c81d8116a4b5d7cd77926b56.js

Limits v/s Request
VPA always works with the requests defined for a container and not the limits. So, the VPA recommendations are also applied to the container requests, and it maintains a limit to request ratio specified for all containers.

For example, if the initial container configuration defines a 100m Memory Request and 300m Memory Limit, then when the VPA target recommendation is 150m Memory, the container Memory Request will be updated to 150m and Memory Limit to 450m.

Selective Container Scaling

If you have a pod with multiple containers and you want to opt-out some of them, you can use the "Off" mode to turn off recommendations for a container.

You can also set containerName: "*" to include all containers.

CODE: https://gist.github.com/velotiotech/af352b64443e1198575c0c377af0d668.js

Conclusion

Both the Horizontal Pod Autoscaler and the Vertical Pod Autoscaler serve different purposes and one can be more useful than the other depending on your application’s requirement.

The HPA can be useful when, for example, your application is serving a large number of lightweight (low resource-consuming) requests. In that case, scaling number of replicas can distribute the workload on each of the pod. The VPA, on the other hand, can be useful when your application serves heavyweight requests, which requires higher resources.

Related Articles:

1. A Practical Guide to Deploying Multi-tier Applications on Google Container Engine (GKE)

2. Know Everything About Spinnaker & How to Deploy Using Kubernetes Engine