Introduction
Businesses always focus on performance and cost. It does not matter if the business is an enterprise, medium sized or a startup company. All of us like to save money and get good deal. With the cloud the cost can add up very quickly and easy. Customers are always looking for automating ways to both advise or help them monitor their cloud usage. Azure provides many resources to help customers monitor their usage.
One of the ways to save money is to shutdown VMs if they are not needed. Azure provides a feature called VM Auto Shutdown. However when we deal with a large scale of VMs count for example if you have thousands of Resource Groups that have hundreds of VMs, it will be very difficult to go to each VM individually and schedule it to shutdown. Of course, you can script it, but then you will have to maintain the script and modify it every time your user wants to change the shutdown time. You will also need to maintain the process for starting the VMs back up.
One of the great Azure offering is Azure Automation. In this PoC we will demonstrate how to use Webapp with REST API to automated Azure Tasks. We will utilize many different Azure services. This demo will focus on start and stop VMs for multi resource groups with multi-schedules or OnDemand.
The source code for this demo can be found here
Getting Started
- Design Diagram
- Software Requirement
- How to install it
- Reference
Design Diagram
The following diagram show how the different services will interact
- Web App: where user can login with their AD login and see their resource groups
- Rest API: It will communicate with automation services
- Key Vault: to store secret keys
- Automation services: It will manage the start and stop VMs based on the defined schedule
- AAD: to authenticate users
Automation Account
The automation account will manage the runbook and schedule and it. Automation account will create Service Principle Account. It is very important to capture this SP.
The Automation Service Principle account must assign contributor role over any subscription where user can schedule stop/start for his VM(s)
We will need to capture SP ApplicationID and Secret and store it in Azure Key Vault
Azure Runbooks
we will need two runbooks.
- One for start VM calls Start-AzureV2VMs and source code can be found under docs\start-script.ps
- One for stop VM calls Stop-AzureV2VMsand source code can be found under docs\stop-script.ps
Azure Key Vault
The key vault will used to store Azure Management API endpoint, Also other sensitive configuration stored there like web app SP for graph and resources, automation account SP. Please refer to README under repo to get full list of required keys and it is expected value.
Azure Container Registry
ACR is required to store the automation api app container and web app container. Please enable username and password so user can user docker to login to the ACR username and password to build container and push the code.
AD App Registration for automation web app
Register AAD app and grant permission for Azure Graph API User read profile. This SP will allow the Web App to get user profile information. Grant also Azure Management resource to allow the webapp to authenticate with AAD. please record the APPID and Secret information to add to Azure Key Vault also to grant access policy to Azure Key Vault.
Azure App Plan for Linux
The resource will host the Web and Rest API apps. so the resource required to be at least 14GB for better performance.
Azure Web App for containers
Automation REST API app: Linux base web app to host the automation API container. Identity must be enabled the system will generate a GUID once Identity is recorded. please record the GUID to enable it with Azure Key Vault. The web API will run using automation SP identity. The web API will read the identity from Azure Key Vault.
The following app settings need to be added:
|
Setting Name |
Description |
|
AUTHENTICATION_ENDPOINT |
|
|
KEY_VAULT |
KV endpoint URL |
|
RESOURCE |
|
|
RESOURCE_KV |
|
|
WEBSITES_PORT |
Web app port default to 5000 |
Automation web app: Linux base web app to host the automation web container. Identity must be enabled the system will generate a GUID once identity is recorded. please record the GUID
To enable it with Azure Key Vault. The web app will run using Graph SP identity. The web API will read the identity from Azure Key Vault.
The following app settings need to be added:
|
Setting Name |
Description |
|
API_VERSION |
1.0 |
|
AUTHENTICATION_ENDPOINT |
|
|
ENCRYPTION_KEY |
Web application auto generation encryption key |
|
KEY |
Web SP for Graph secret key -- it will be removed in future release |
|
CLIENT |
Web SP for Graph AppID -- it will be removed in future release |
|
KEY_VAULT |
KV endpoint URL |
|
OAUTHLIB_INSECURE_TRANSPORT |
True |
|
OAUTHLIB_RELAX_TOKEN_SCOPE |
True |
|
RESOURCE |
|
|
RESOURCE_GRAPH |
|
|
RESOURCE_KV |
|
|
REST_API_ENDPOINT |
Automation REST API web app URL |
|
SUBSCRIPTION_ID |
Web App Subscription |
|
TENANT |
<tenantname>.onmicrosoft.com |
|
TENANT_ID |
your tenant ID |
|
WEBSITES_PORT |
Web app port default to 8000 |
Once the web app is generated please record the web url and modify the AAD SP for web app by adding the web app url under "Redirect URLS"
Azure Storage
Azure storage Account will be needed to enable log stream.
Software Requirement
- REST API will be built in Python and container after
- Web App will be in Python Django
- App plans for Linux
How to install it
1- Make sure the assets installed and configured
2- All the app setting and key vault secrets in place
3- Log into Azure ACR using `docker login` and provide the username and password.
4- clone git repo
5- change directory to "src\api" and run docker build file "docker build -t <ACR URL>\<automation-api-web-name>:tag . "
6- Run docker push "docker push <ACR URL>\<automation-api-web-name>:tag"
7- From the portal go to webapp instance and modify the Container Setting to reflect the container name and tag.
8- Restart the WebAPP / Web API
9- In many case clear the cache to see the new changes.
Reference