Tip: running a cheap AKS lab
Published May 21 2020 03:55 AM 5,256 Views

I am currently running three AKS labs in one of my subscriptions and I constantly try to minimize costs so I'm quite regularly spinning a "cluster" with only one worker node (Standard DS2 V2) which I stop whenever I don't use it. Lately, when working with KEDA, I enabled the Virtual Nodes (see below) when creating the cluster:



which makes use of Azure Container Instances (ACI) behind the scenes. Then I realized that, I could leverage Virtual Nodes more often in order to reduce costs. Therefore, I thought of spinning up the cluster with the smallest possible Virtual Machine as the main worker node.


While  virtual machines represent a fixed cost, the pricing model of ACIs is based on actual consumption as they are only charged per second of execution which may be just enough for test scenarios/demos/lab-kind of activities. By default, if you do not specify any resource request in your container, AKS will provision ACIs with 1 CPU and 1.5 GB of memory. When specifying resources, for instance like this:




apiVersion: apps/v1
kind: Deployment
    app.kubernetes.io/name: serverless
    app.kubernetes.io/part-of: serverless
    app.kubernetes.io/version: v9
  name: serverless
  namespace: default
  replicas: 1
      app: serverless
        app: serverless     
      - name: serverless       
        image: akspremiumregistry.azurecr.io/serverlesscontainer:1
          memory: "4000Mi"
          cpu: "4"       
       - name: regsecret
      - key: virtual-kubelet.io/provider
        operator: Equal
        value: azure
        effect: NoSchedule




, AKS applies the requested resources to the ACI config as illustrated here:



Benefits of such a setup

  • Cost-friendly, providing you do not let ACIs hanging around
  • Allows for resource-greedy scenarios 


Limitations of such a setup


Although interesting from a financial perspective, it comes with a few limitations:

  • Suitable for lab-only activities
  • Suitable for short-running containers (again, perfect in a demo scenario, etc.). When used like this, you might end up with only a few euros/dollars to pay at the end of the month instead of much more for a full blown worker node.
  • Container startup is a little delayed because of the ACI provisioning.
  • By default, only 100 concurrent ACIs are allowed per subscription, meaning that you can't have more than 100 concurrent pods. This can be changed by reaching out to support.
  • Make sure to clean your stuff after use, else the ACIs will keep running forever which then will become very costly. The easiest way is to run a kubectl delete namespace .... but the bare minimum is to run a kubectl scale deploy --all --replicas=0 so as to let AKS destroy the corresponding ACIs. 

To wrap it up, to get a cheap AKS lab, just proceed the following steps:

  • Create the cluster & enable Virtual Nodes
  • Choose the smallest possible VM as your main worker node


1 Comment
Version history
Last update:
‎May 22 2020 09:44 AM
Updated by: