Over the last two “Best Practices” posts, I’ve looked at how to Plan and Build a Hybrid Cloud, and with these technical exercises complete, this post will focus on the best practices for deploying this carefully planned and built Hybrid Environment.
In this post, I’ll examine some critical items for deployment, like Service Provider Foundation and Windows Azure Pack functionalities like IaaS ( Software Defined Networking , Remote Console Setup , Gallery items, VM Templates), Websites (use of Proxy Servers, offline Gallery), and Databases (Microsoft SQL Server, MySQL). I’ll also be looking at vital functions like Service Management Automation , Usage, WAP Authentication Providers, Portal Theming/Customization, and Migration.
I’ll also identify how to troubleshoot some of the most common deployment obstacles. Ready to dig in?
Service Provider Foundation
Service Provider Foundation (SPF) comes as part of System Center - Orchestrator . SPF is an extensible OData web service that interacts with Virtual Machine Manager , Operations Manager , and Orchestrator (among other things) thus enabling these products to be used in a multi-tenant environment such as a Service Provider. In the Cloud OS vision , SPF is key in the communication between Windows Azure Pack and Virtual Machine Manager – but it is also leveraged for the usage consumption pipe enabled with Operations Manager .
Also of note: There is a SPF Server entry for the Remote Console that is part of IaaS, and SPF itself supports up to five VMM stamps.
The best practices for the setup and operation of Service Provider Foundation can be complex without some insight, so I’ll begin with six key pitfalls to avoid during setup:
- SPF IIS App Pool needs to run with a Domain Account that is also a VMM Administrator.
- This Domain Account must be a member of the 4 local SPF Groups.
- The user of this account needs to register SPF in Windows Azure Pack, otherwise Service Management Automation may encounter issue when attaching Runbooks to SPF Action. A local user can be used if no Runbooks are attached to SPF Actions, however.
- This account also needs access to the SQL Server that you specify during deployment.
- Configure IIS on the SPF machine with Basic Authentication.
- Login to the SPF server once with the domain service account.
To get deeper on these particular elements, I recommend reading:
- How to install Service Provider Foundation .
- General Troubleshooting List for Windows Azure Pack.
- Configure SPF for IaaS usage and metering .
- TechEd 2013: Multi_Tenant IaaS Clouds in Microsoft System Center and Windows Server .
Windows Azure Pack
Windows Azure Pack (WAP) for Windows Server is a collection of Windows Azure technologies that run on top of Windows Server 2012 R2 & System Center 2012 R2 and enable a consistent cloud experience across public, private and hybrid clouds.
I’ve posted many times before on the importance of consistency across your clouds, and, while I don’t want to beat on too much again, it simply cannot be overemphasized that Microsoft is the only organization in the world operating an at-scale, global Public Cloud and then taking what we learn and delivering it for you to use in your datacenter. WAP is concrete evidence of this work.
For the purposes of Deployment, I’m going to focus on three specific resources delivered by WAP: IaaS, Websites, and Databases. If you aren’t familiar with WAP (or if you don’t have it already), I recommend reading this deployment overview for WAP and Windows Server, and this overview of WAP installation and configuration .
IaaS
Windows Azure Pack is a core component of the infrastructure-as-a-service capabilities we deliver through Windows Server and System Center. Our IaaS capabilities allow you to host Windows and Linux virtual machines in a cloud architecture in your datacenters. These capabilities also include a VM Gallery, scaling options, VM access options and virtual networks. To learn more about the specifics of Microsoft’s IaaS offering, I recommend this article .
Three important elements of the IaaS features include SDN, Remote Console, Gallery items, and VM templates.
Software-defined NetworkingA major milestone in networking is the platform capability of an inbox NVGRE Gateway to bridge communication from a VM/Tenant Network to networks outside of the virtualized network. Check out these links for more information on the virtual network capabilities delivered through the platform and how they work within WAP.
- Hybrid Clouds using Hyper-V Network Virtualization: Part 1 , Part 2 , Part 3 .
- Web Platform Installer Feed (containing the NVGRE Gateway VMM Template).
- Also check out Microsoft’s SDN strategy .
When deploying the HA NVGRE Gateway Service Template available as a free download via the Web Platform installer, keep in mind these three things:
- A host cluster is required for placing the HA Gateway Service (PA Address).
- Don’t make the Gateway VMs highly available because if this occurs the CA Address will no longer follow PA Address.
- Don’t configure your virtual switch manually with absolute QoS mode.
One of the improvements to Virtual Machines is the ability to get a direct connection to the console session. There are a lot of scenarios where this is hugely important, and one in particular is addressing a situation where a tenant miss-configures the network settings and can no longer connect via Remote Desktop. Instead of going through the process of opening a support ticket, the tenant can now fix the problem independently by using the new console connect option.
This diagram outlines the components required when accessing the VM Console from an untrusted network.
You can find detailed setup, installation, and configuration instructions in this guide .
Certificate requirements seem to cause some confusion in this configuration, so let’s examine this for a moment: The certificate used to sign the token between VMM, RD Gateway Plugin and the Hyper-V Host is different from the certificate used to sign the RDP file that gets downloaded in the tenant portal and opened by the client computer. The certificate to sign the RDP file must have the FQDN of the RD Gateway as CN.
Gallery ItemsWindows Azure Pack includes a VM Gallery that contains VM Roles . A VM Role technically consists of two parts:
- Resource Definition package : Forms the basis for creating a virtual machine.
- Resource Extension package : Forms the basis for installing applications on a virtual machine.
Part of this configuration of VM Roles requires you to assign specific tags to virtual hard disk images. This configuration also requires the Operating System, Family Name, and Version of the virtual hard disk to be specified. To make this process straightforward, each VM Role example comes with a deployment guide that outlines these requirements. There is also a wide range of example gallery items currently available through the Web Platform Installer , but you can also build your own Gallery Items using the VM Role Authoring Tool .
To get much deeper on this topic, check out these additional posts from the engineers who’ve built these features:
- Enabling On-Prem IaaS Solutions with WAP .
- Troubleshooting Gallery Items: Part 1 , Part 2
- Using Gallery Items in Virtual Machine Clouds .
- Overview of VM Role Authoring Tool .
- Web Platform Installer Feed (containing example Gallery Items).
The website component of Windows Azure Pack provides high-density, multi-tenant web hosting services. This is a scalable, shared, and secured web hosting platform for template-based web applications and programming languages like ASP.NET, PHP, and Node.js. This is a capability we innovated in Azure, proved in Azure, and we have now delivered it for you to run in your datacenters. With this functionality you can run 5,000 web sites on a single Windows Server OS instance. To deploy this component, visit this page .
Use of Proxy ServersHave you ever tried to create a new website based on a Gallery item but found the list empty? The cause of this is very likely that you are blocked by your proxy server. The typical troubleshooting process for this includes checking the proxy server log files and then verifying if you can reach the gallery URL in a web browser on the machine that has the Web Application Gallery component installed.
Offline Gallery
If you are working in a secure environment where you want to control which web application gallery items are available to your tenants, you may consider using an offline copy for the Gallery. This arrangement also allows you to do code reviews and approve the gallery items. For an in-depth overview of the necessary steps to do this, check out this detailed post .
File Server
Windows Azure Pack Web Sites requires a File Server. This can be a standalone File Server, File Server Cluster, or a third party NAS device. For more insights about all the requirements, check out this article .
Databases
Windows Azure Pack has the capability to support Microsoft SQL Server or MySQL Database hosting for tenants or, Database as a Service (DbaaS). These databases are often used in conjunction with Web Sites Services , and are also offered as part of the Windows Azure Pack. To learn more about installing and configuring the SQL Server and MySQL resource providers, checkout this overview . Also note that for this database functionality you must first license and deploy instances of MySQL or SQL Server outside of WAP (in this way, WAP is used as a means to provision database services).
To provide high availability for your tenant databases you can use SQL AlwaysOn Availability Groups . SQL AlwaysOn enables you to use Azure as an extension of your datacenter for backup and disaster recover or your databases – a very cool and easy to use hybrid cloud scenario. This feature is part of SQL Server Enterprise Edition . With SQL Server 2012 these backup/DR scenarios are available with manual scripting provided by the SQL engineering team on MSDN; with SQL Server 2014 these scenarios will be much easier with a UI that is built into SQL Server Management Studio.
You can read a lot more about this on these TechNet posts:
- Configure SQL AlwaysOn Availability Groups in Windows Azure.
- Using SQL Server 2012 AlwaysOn Availability Groups with System Center 2012 – Ops Mgr .
- Enable and Disable AlwaysOn Availability Groups.
- Setting up AlwaysOn Availability group on SQL Server 2012 .
The latest version of MySQL can be obtained and installed via the Web Platform Installer . The configuration step that is specific to MySQL is to enable remote login. If you miss that important step you will not be able to register MySQL Servers in the Admin Portal.
Service Management AutomationService Management Automation (SMA) is a new component that comes as part of System Center (available on the Orchestrator image). SMA enables you to perform automation in the cloud, and, like SPF, SMA exposes an extensible OData web service. SMA leverages Runbooks to enable automation in the Windows Azure Pack. SMA Runbooks are Windows PowerShell Workflow scripts, which can be imported and/or authored right within the Windows Azure Pack. Runbook execution can be scheduled, triggered by a WAP/SPF event, or manually initiated. With this in mind, one of the primary uses of Automation within WAP is to execute Runbooks based on other WAP actions, e.g. starting a Virtual Machine.
A couple noteworthy best practices for setting up and operating Service Management Automation are:
- Like SPF, SMA IIS App Pool needs to run with a Domain Account.
- In order to execute Runbooks that use SPF, certificates need to be issued for the SMA server and SPF server which trust each other.
- Runbooks needs to be “tagged” in order to be used for automation (e.g. to show up under VM Automation, a Runbook needs to be tagged with SPF).
For further reading on this particularly complex topic, I recommend the following links:
- Detailed overview and introduction to SMA .
- How to install the Service Management Automation Web Service .
- Calling an Orchestrator Runbook from SMA, Part 1 .
- Service Management Automation: SMA Runbook Toolkit Spotlight–SMART for Runbook Import and Export .
Tracking Cloud Usage is fundamental. Pairing System Center with Windows Azure Pack offers one of the best ways to deliver usage statistics that enable pay-as-you-go scenarios for the services in WAP. Usage in WAP leverages the following components: Virtual Machine Manager , Operations Manager , Service Provider Foundation and Service Reporting .
With these components, partners and enterprises can extract usage data from Windows Azure Pack by using an OData web service. This then offers three scenarios for usage:
- Chargeback by using CloudCruiser .
- Service Reporting using SQL.
- Building a custom billing adapter.
Some straightforward best practices for setup and operation usage are:
- Operations Manager, Virtual Machine Manger and Service Provider foundation must be in place to enable VM Clouds usage.
- The WAP Usage Service must have a User and Password set before data can be extracted.
- SPF IIS App Pool needs to have read access to Operations Manager Data Warehouse .
- Service Reporting needs access to Operations Manager Operational Database .
- CloudCruiser extension enables chargeback on top of Windows Azure Pack.
For further reading, I recommend the following links:
- IaaS Usage and Service Reporting using System Center 2012 R2 and Windows Azure Pack.
- Configuring VMM and OM for IaaS usage and metering .
- Installing & configuring Service Reporting for IaaS usage and metering .
- Troubleshooting Windows Azure Pack & Usage .
Windows Azure Pack supports multi-tenant authentication by using claims-based authentication . This offers a flexible way to authenticate users logging into Windows Azure Pack by providing support for a wide range of authentication technologies like ADFS, SAML, WS and others. Once authenticated a user will be given access to (and can then consume) services within WAP based on assigned subscriptions. By default the WAP Tenant uses .Net authentication , but can easily be changed to use other authentication providers. The WAP Admin Portal uses Windows Authentication by default, but this can also be changed to use ADFS.
Authentication in WAP allows you to do the following two things:
- Provide administrative access to users from its own Active Directory.
- Provide self-service access to the Tenant Portal to users from a tenant.
Some noteworthy best practices for the setup and operation of authentications are:
- Change authentication service in WAP to use CA certificates .
- Configure ADFS to authenticate with WAP.
- Configure ADFS to authenticate with tenants ADFS or other claims-based services.
- Create users in the WAP portal.
- Have users login with their own credentials in the WAP portal and provide them access to WAP.
For further reading I recommend the following links:
- Claims-Based Identity in Windows Azure Pack.
- Federated Identities to Windows Azure Pack through AD FS - Part 1 , Part 2 , Part 3 .
- Creating users in Windows Azure Pack .
- AD FS and the AD FS Console .
Windows Azure Pack can be modified to suit partner/enterprise/service provider needs in three primary ways:
- Changing the theme.
- Building a Resource provider.
- Building your own portal.
The portal theme allows simple modification of the portal by customizing the tenant user experience to include custom logos, colors, and icons.
Two best practices to keep in mind for portal theming are:
- Take advantage of the sample theme kit provided with the Windows Azure Pack Developers Kit . This kit demonstrates areas of the tenant portal that can be customized.
- The Contoso theming kit shows images and styles consistent with the branding of an imaginary hosting company named Contoso
For more information, check out these WAP Service Management API Samples .
MigrationThere are two key Migration scenarios.
- Migration from Windows Azure Services for Windows Server .
- Integration of WAP to an existing System Center deployment. This scenario targets IaaS workload only.
If you made a bet on Windows Azure Services for Windows Server when it was released earlier this year, definitely take the time to get the newer (and free!) Windows Azure Pack to start taking advantage of the new features. The detailed step by step guide can be found in the link below.
The second scenario mentioned above is all about ensuring existing Virtual Machines show up in WAP Tenant Portal as expected, owned by the appropriate tenant user role.
A couple articles worth noting on this topic:
- Migrate from Windows Azure Services for Windows Server .
- Adding an Already Running VM in VMM to a WAP Subscription .
* * * *
This topic is obviously very complex, and for ongoing information, best practices, and up-to-date knowledge, I highly recommend the Deployment track over on the Building Clouds blog .
After putting this post together I have got to admit that there is some work we need to do to simplify, simplify, simplify .
In the next post we tackle everything that comes after your hybrid environment is deployed: Operation and Management.